1

Reactive Systems

Yolanda Gil

CS 541, Fall 2003

(Thanks to Karen Myers from SRI International)

2

The problem with plans (I)

Attack Goliath1. Gather pile of rocks

2. Grasp slingshot

3. Fire at giant

4. Hit on the head

3

The problem with plans (II)

Attack Goliath1. Gather pile of rocks

2. Grasp slingshot

3. Fire at giant

4. Hit on the head

• Unknown how many stones

• Unknown if stones

• Unknown how many attempts

• Conditions for termination

• What if failure

• Check state

4

Reactive Systems• Embedded in the real world• Have sensors and effectors• Actively test the external environment• Need to respond to events in dynamic

environments• Failure may require aborting and generating

new response• Do we need deliberate reasoning (planning)?

5

Outline and Informal Roadmap

• Control systems– Networks of “variables” (arcs) and “functions” (nodes)

• Reactive Action Packages (RAPs)– Networks of “conditions” and “tasks”

• Task Control Architecture (TCA)– Network arranged according to “vertical capabilities”

• Procedural Reasoning System (PRS)– Integrates planning, BDI, and reactive techniques

• Anytime algorithms– When time is short, managing what you think about

• Other approaches and issues

6

Readings

• RAP (http://people.cs.uchicago.edu/~firby/raps)

– Firby, J “Task Networks for Controlling Continuous Processes”, Proceedings of Artificial Intelligence Planning conference, 1994.

• TCA (http://www-2.cs.cmu.edu/afs/cs/project/TCA/release/tca.orig.html, http://www-2.cs.cmu.edu/afs/cs/project/TCA/release/tca.html)

– Simmons, R. “Structured Control for Autonomous Robots”, IEEE Transactions on Robotics and Automation, Feb 1994.

• PRS (http://www.ai.sri.com/~prs)

– Reactive reasoning and planning: an experiment with a mobile robot, M. Georgeff and A. Lansky, in Proceedings of AAAI, 1987.

• Anytime algorithms– Zilberstein, S. “Using Anytime Algorithms in Intelligent

Systems”, AI Magazine, 1996.

7

Control Systems: An Example (I)Control of temperature profile for a spray deposition process. Jones, P.D.A.; Duncan, S.R.; Rayment, T.; Grant, P.S. IEEE transactions on control systems technology special issue on control of industrial spatially distributed processes, Sept 2003.

8

Control Systems: An Example (II)Control of temperature profile for a spray deposition process. Jones, P.D.A.; Duncan, S.R.; Rayment, T.; Grant, P.S. IEEE transactions on control systems technology special issue on control of industrial spatially distributed processes, Sept 2003.

9

Beyond Stimulus-Response

• Address problems that require a combination of:

– Coordinated activity to accomplish tasks

– Reactivity to world dynamics

• Situate control decisions within an explicit, persistent decision-making framework

10

Reactive Action Packages (RAP)

11

A Symbolic Discrete Task

12

Waiting for a signal to proceed

13

Concurrent tasks

14

More Complex Task Networks

15

Task Control Architecture (TCA)

• Vertical task decomposition: several task-specific modules communicate through a central control module

• Deliberation: top-down task-subtask, resolve constraints

• Central control routes messages– Inform, query, command, monitor

16

Ambler Walking Robot

17

Ambler Modules

18

Ambler Task Tree

19

TCA: Monitoring

• Central control traverses tree and handles messages: – asks gait planner to traverse arc, – gait planner asks terrain mapper for elevation

map in order to take steps– Gait planner asks leg recovery planner to place

leg, move leg, move body,– Gait planner activates monitor whether

achieved position

20

TCA: Control

• Ordering and temporal constraints• Delay planning constraint: goal cannot be issued

until previous task achieved– Can do place leg planning while monitoring achieve

position

• Exception handling: error recovery modules examine and modify task trees– Eg: if position not achieved, add take steps subtask

21

Ambler Planning and Execution

22

An Alternative to TCA’s Vertical Capabilities:Horizontal Layered Control

Reason about behavior of objects

Plan changes to the world

Identify objects

Monitor changes

Build maps

Explore

Wander

Avoid objects

23

Procedural Reasoning System (PRS)

• Framework for symbolic reactive control systems in dynamic environments– Eg Mobile robot control– Eg diagnosis of the Space Shuttle’s Reaction

Controls System

24

PRS: Main Features

• Pre-compiled procedural knowledge• BDI (Belief, Desires, Intentions) foundation• Combines deliberative and reactive features

– Plan selection, formation, execution, sensing

• Plans dynamically and incrementally• Integrates goal-directed and event-driven behavior• Can interrupt plan execution• Meta-level reasoning• Multi-agent planning

25

PRS Architecture

Interpreter

Tasks Procedures

IntentionsDatabase

User

World

26

PRS Architecture: Database

• Contains beliefs or facts about the world

• Includes meta-level information– Eg goal G is active

Interpreter

Tasks Procedures

IntentionsDatabase

User

World

27

PRS Architecture: Tasks

• Represent desired behavior

• Conditions over some time interval– eg (walk a b): set of

behaviors in which agent walks from a to b)

Interpreter

Tasks Procedures

IntentionsDatabase

User

World

28

Expressing Tasks in a Dynamic Environment

• (! P) -- achieve P

• (? P) -- test P

• (# P) -- maintain P

• (^ C) -- wait until C

• (-> C) -- assert C

• (~> C) -- retract C

29

PRS Architecture: Intentions

• Currently active procedures

• Procedure currently being executed

Interpreter

Tasks Procedures

IntentionsDatabase

User

World

30

PRS Architecture: Procedures

• Pre-compiled procedures

• Express actions and tests to achieve goals or to react to conditions

Interpreter

Tasks Procedures

IntentionsDatabase

User

World

31

• Environment conditions – Purpose (goal or condition)– applicability criteria

• Plot– directed graph– partially ordered conditional &

parallel actions, loops– Successful node execution by

achievement of node’s goals– If no body: primitive action

Metapredicates– Achieve – Achieve-By {proc}

– Test – Conclude {effects}

– Wait-Until – Use-Resource– Require-Until

Cross-Country Delivery

Cue:

(ACHIEVE (DELIVER CUSTOMER.1 GOODS.1))

Preconditions:

(TEST (AND (LOCATED CUSTOMER.1 CITY.2) (LOCATED GOODS.1 CITY.1)

(DISTANCE CITY.1 CITY.2 DISTANCE.1) (> DISTANCE.1 1000) ) )

Setting:

(TEST (AND (AIR-SHIPMENT AIRCARGO.1 GOODS.1) (LAND-SHIPMENT LANDCARGO.1 GOODS.1) ) )

Resources:

- no entry -

Propertities:

(AUTHORING-SYSTEM ACT-EDITOR)

Comment:

Long distance delivery of goods to customers

(ACHIEVE (RECORD-INVOICE CUSTOMER.1 GOODS.1 INVOICE.1) )

(ACHIEVE (LOCAL-DELIVERY CUSTOMER.1 GOODS.1) )(CONCLUDE (COMPLETED-INVOICE INVOICE.1) )

(ACHIEVE-BY (LOCATED LANDCARGO.1 CITY.2) SHIP-BY-RAIL) )

(ACHIEVE-BY (LOCATED AIRCARGO.1 CITY.2) SHIP-BY-AIR) )

Representing Procedures with Act Formalism

32

PRS InterpreterExecution Cycle

1. New information arrives that updates facts and/or tasks

2. Acts are triggered by new facts or tasks

3. A triggered Act is intended

4. An intended Act is selected

5. That intention is activated

6. An action is performed

7. New facts or tasks are posted

8. Intentions are updated

Goal2ACT8sleeping

Fact1ACT2normal

Goal3ACT3sleeping

Intention Graph

Cue: (TEST (overpressurized tank.1))

ACT2

Act Library

Act Execution

(overpressurized fuel-tank)

(ACHIEVE (position ox-valve closed))

New Facts & Tasks

ExternalWorld

1

2

3

4

5

6

7

8

Cue: (ACHIEVE (position valve.1 closed))

ACT1Facts&

Tasks

(ACHIEVE (position ox-valve closed))

ACT1current

33

Meta-Reasoning

• Can include meta-level procedures– eg: choose among multiple applicable

procedures– eg: evaluate how much more reasoning can be

done within time constraints– eg: how to achieve a conjunction or disjunction

of goals

RCS Jets

RCS Controls

PT

Rlv Valve

HETank

A BValve

Jet

Regulator

12 345

1 2 3 4 5

T Temp

P P P P

P

T

P P

P Pressure

P

FUTank

Talkback

Switch

A B

Control Panel

12 345

1 2 3 4 5

OP OP OPOPOP

OP CL

OP CL

opengpc

close

opengpc

close

Jet Fail - OnCue Test:

Alarm sounding,RCS warning light on,Status RCS jet.1 is failed-on,GPC displays dir.1 for jet.1 for rcs.1

Preconditions Test:

Direction jet.1 is dir.1

Setting Test:

Connected manifold.ox to jet.1,Connected manifold.fu to jet.1,Connects valve.fu by leg.fu

to manifold.fu,Connects valve.ox by leg.ox

to manifold.ox,Oxidizer-subsystem ox.1 of rcs.1,Fuel-subsystem fu.1 of rcs.1,Part valve.ox of ox.1,Part valve.fu of fu.1

Achieve:Position valve.ox closed,Position valve.fu closed

Achieve:Notify "Thruster jet.1 failed-on"

Test:High-usage of jet.1

Test:Not high-usage of jet.1

Achieve:Notify "Thruster jet.1 failed-on

ELECTRICALLY"

Test:Type jet.1 vernier

Test:Not type jet.1 vernier

Achieve:Notify "Thruster jet.1 failed-on

INPUT CARD"

Achieve:Notify "TURN-OFF rcs.1 manifold.ox

& manifold.fu DRIVER"

Achieve:Pressure manifold.ox is pres.ox,Pressure manifold.fu is pres.fu

Test:> pres.ox 130,> pres.fu 130

Test:≤ pres.ox 130,≤ pres.fu 130

FACTS&

BELIEFS

ExternalTASKS

ExternalFACTS

Executing procedures can post

GOALS, FACTS, & BELIEFS

orsend MESSAGES

TASKS

Procedure Library

Jet Fail - On

Jet Fail - On

Determine new procedures

that are eligiblefor execution

Select procedures for execution

Dump Propellant

Regulator Test

Regulator Test

ShuttleGPC

MESSAGES

Shuttle’s RCS Malfunction Handling

• Automates specification and execution of RCS malfunction procedures.

• Reacts to changes in RCS. Ensures safe operation while carrying out diagnosis and remediation procedures.

35

Multiple Tasks, Multiple Agenst

• Multithreaded operation: multiple tasks being performed, runtime stacks where tasks are executed, suspended, and resumed

• Supports distributed planning: several PRS agents run asynchronously and communicate through message passing

36

Anytime Algorithms

• Time to deliberate about events varies

• Algorithms to compute the best answers they can in the time available

• Anytime algorithms– Can be suspended and resumed with little overhead

– Can be terminated at any time and return some answer

– The answers returned improve with time

37

A time-dependent planning problem

• Observe (O)• React (E): time required to carry out reaction

of type E• Herald (C): earliest observation time that

enables prediction of condition C requiring a response

• Utility (C,E): utility of reacting to with E to C• Response (C): time between having

information to predict C and C occurring

38

When Time is Short…

• Prediction time: time required to predict event given info available

• Deliberation time: max time for committing to a reaction (if reaction is needed)

• Reaction time: time required to react to event– React(E) + Response(C)

39

Deliberation

• Decision procedure D for each C: given t time to deliberate, D returns best guess E about how to react

• Utility(C, D(C,T))

• Deliberation scheduling:– Given several deliberation procedures,

determine how to best allocate deliberation time

40

Utility versus time

One-shot improvement Linear improvement, bounded utility

Linear improvement, unbounded utility

Diminishing returns

41

Other Approaches and Issues

• Blackboard architectures (Guardian)• Universal plans• Related issues covered in the course:

– Reasoning about uncertainty– Learning

• from the environment• Becoming increasingly reactive

42

Summary

• Control systems– Networks of “variables” (arcs) and “functions” (nodes)

• Reactive Action Packages (RAPs)– Networks of “conditions” and “tasks”

• Task Control Architecture (TCA)– Network arranged according to “vertical capabilities”

• Procedural Reasoning System (PRS)– Integrates planning, BDI, and reactive techniques

• Anytime algorithms– When time is short, managing what you think about

• Learning and uncertainty reasoning