Download - Presented by: Robert T. Effinger
May 12, 2004
Massachusetts Institute of Technology
Presented by: Robert T. Effinger
January 25, 2005
Qualifying Exam Research Presentation
May 12, 2004
Massachusetts Institute of Technology
“Enabling Fast Flexible Planning through Incremental Temporal Reasoning with
Conflict Extraction”
I’shiang Shu, Robert Effinger, Prof. Brian WilliamsModel-Based Embedded and Robotic Systems Group
2005 International Conference on Automated Planning and Scheduling
3
Presentation Summary
• Importance of Automated Planning Research
• Three Main Types of Automated Planners– A Simple Example– Generative Planner– Path Planner– Task Network Planner
• Research Objectives of the Kirk Planner– Fast and Flexible Planning
• Key Ideas and Concepts of the Kirk Planner– Simple Temporal Network (STN)– Incremental Update Algorithm (ITC)
• Experimental Data
• Key Assumptions and Limitations
• Conclusions
4
Importance of Automated Planning
• Airline Timetables• Assembly Lines• Radio Frequency Planning• Military Convoys• ISS Life Support Rations• MER Rover Planning • NASA’s Deep Space Probes
5
Three Main Types of Automated Planners
– Generative Planners
– Path Planners
– Task Network Planners
6
A Simple Planning Example
Football Game:
20
10
G
QB
C R
Goal: Touchdown
Start Condition: 20 yardline
Library of Actions: Center (C):
- Hike- Block
Reciever (R):- Run to Endzone- Run Slant- Catch Football
QuarterBack (QB)- Pass to Endzone- Slant Pass- QB Sneak
7
Generative Planner
Football Game:
20
10
G
QB
R
Goal: Touchdown
Start Condition: 20 yardline
Library of Actions: Center (C):
- Hike- Block
Reciever (QB):- Run to Endzone- Run Slant- Catch Football
QuarterBack (QB)- Pass to Endzone- Slant Pass- QB Sneak
20 yardline
Generate Forward until Goal Condition Found: (Graphplan, FF)
(C) Hike
Start Condition: (R) Run to Endzone
(QB) Pass to Endzone(R) Catch Football
Goal:
Generate Backward until Start Condition Found: (PoP)
Touchdown !
C
Generative Planning Advantage: Very Expressive
Generative Planning Disadvantage: Often Intractable (exponential growth) Can Generate Infeasible Plans
Activity Times are not Flexible
Path Planner
8
Task Network Planner
Football Game:
20
10
G
QB
C R
Start Condition: 20 yardline
Goal: Touchdown
Library of Actions: Center (C):
- Hike- Block
Reciever (QB):- Run to Endzone- Run Slant- Catch Football
QuarterBack (QB)- Pass to Endzone- Slant Pass- QB Sneak
9
Task Network Planning
Goal: Touchdown
Start Condition: 20 yardline
Library of Actions: Center (C):
- Hike- Block
Reciever (QB):- Run to Endzone- Run Slant- Catch Football
QuarterBack (QB)- Pass to Endzone- Slant Pass- QB Sneak
Activity Plans:
20
10
G
QB
C R
Hail Mary Slant Pass
20
10
G
QB
C R
20
10
G
QB
C R
QB Sneak
20 yardline
Hail Mary Activity Plan:
(C) Hike(R) Run to Endzone
(QB) Pass to Endzone(R) Catch Football
Touchdown !
10
Task Network Planning
Goal: Touchdown
Start Condition: 20 yardline
Library of Actions: Center (C):
- Hike- Block
Reciever (QB):- Run to Endzone- Run Slant- Catch Football
QuarterBack (QB)- Pass to Endzone- Slant Pass- QB Sneak
Activity Plans:
20
10
G
QB
C R
Hail Mary Slant Pass
20
10
G
QB
C R
20
10
G
QB
C R
QB Sneak
Task Network Advantages: Tractable in real-time Activity times can be flexible
Task Network Disadvantage: Less expressive
11
Research Objectives of the Kirk Planner
“To Enable Fast and Temporally Flexible Planning”
Translates into two technical research objectives:
2.) When replanning, try to reuse previous planning computations
1.) Allow flexible instead of fixed execution times on activities
Kirk’s Main Research Objective:
12
Two Key Ideas
“To Enable Fast and Temporally Flexible Planning”
2.) When replanning, try to reuse previous planning computations
1.) Allow flexible instead of fixed execution times on activities
Kirk’s Main Research Objective:
- Kirk supports a lower and upper time-bound on each activity
- an incremental update algorithm (ITC) that utilizes previous computations
Translates into two technical research objectives:
13
1st Key Idea: Flexible Execution Times
A Simple Temporal Network (STN):
Kirk supports flexible lower and upper time-bounds on activities
Touchdown !
20 yardline (C) Hike(R) Run to Endzone
(R) Catch Football(QB) Pass to Endzone
Another Representation of the Hail Mary Activity Plan:
20
10
G
QB
C R
Hail Mary:
(C) Hike [0,5]
( Touchdown ! )
(QB) Pass to Endzone [5,8]
(R) Run to Endzone [7,9]
(R) Catch Football [0,2]
14
Determining Temporal Consistency
To determine temporal consistency, an STN is converted into a distance graph
(C) Hike [0,5]
(20 yardline) ( Touchdown ! )
(QB) Pass to Endzone [5,8]
(R) Run to Endzone [7,9]
(R) Catch Football [0,2]
STN:
Distance Graph:
5
-0
Then the FIFO Label Correcting Algorithm is run on the new distance graph.
There are two possible outcomes:
1.) If the algorithm terminates, then the STN is temporally consistent
2.) If the algorithm enters an infinite loop, then the STN is temporally inconsistent - an infinite loop is detected as soon as a nodes cost drops below zero
- the infinite loop will contain the timing constraints in conflict.
-0
0
-00
8
-5
9
-7
0
-00
-0
2
-0
15
0
14
5
155
5
13
13
13
5
0
0
0
00
8
-5
9
-7
0
00
0
2
0
Determining Temporal Consistency
Update
FIFO Label-Correcting Algorithm(Graph G)
{01} for all s V(G) {02} d(s) = ∞{03a} d(sstart) = 0{03b} insert(Sstart){04a} while !Q.empty(){04b} u = Q.pop()
{05a} for v Succ(u){05b} dval = Update(u,v){06} if(dval) < 0{07} return false;{08} return true;
value Update(p,x){09} if (d(x) > d(p) + c(p,x)){10} d(x) := d(p) + c(p,x);{11} Q.Insert(x, d(x));{12} return d(x);
Check for Violating Arcs
Initialize
Run FIFO Label-Correcting Algorithm:
Original STN:
(C) Hike [0,5]
(20 yardline) ( Touchdown ! )
(QB) Pass to Endzone [5,8]
(R) Run to Endzone [7,9]
(R) Catch Football [0,2]
Terminates, so STN is consistent!!
16
0
20
5
5
5
13
1313
5
0
0
0
00
8
-5
9
-7
0
00
0
2
0
Detecting a Conflict
Update
FIFO Label-Correcting Algorithm(Graph G)
{01} for all s V(G) {02} d(s) = ∞{03a} d(sstart) = 0{03b} insert(Sstart){04a} while !Q.empty(){04b} u = Q.pop()
{05a} for v Succ(u){05b} dval = Update(u,v){06} if(dval) < 0{07} return false;{08} return true;
value Update(p,x){09} if (d(x) > d(p) + c(p,x)){10} d(x) := d(p) + c(p,x);{11} Q.Insert(x, d(x));{12} return d(x);
Check for Violating Arcs
InitializeRun FIFO Label-Correcting Algorithm:
Modified STN:
(C) Hike [0,5]
(20 yardline) ( Touchdown ! )
(QB) Pass to Endzone [5,8]
(R) Run to Endzone [7,9]
(R) Catch Football [0,2]
Node cost dropped below zero! STN is Inconsistent !!
(R) Walk to Endzone [12,15]
-12
15
1
1
1 9
9
9-3
Following the backpointers reveals conflict!
17
2nd Key Idea: Incremental Update Algorithm (ITC)
Three Arc Update Rules:
1.) Arc Change With No Effect to Shortest-Path
2.) Arc Change Improves Shortest-Path
3.) Arc Change Invalidates Shortest-Path
Use an Incremental Update Algorithm (ITC) to retain previous computations
0
14
5
155
5
13
13
13
5
0
0
0
00
8
-5
9
-7
0
00
0
2
0
Hail Mary Distance Graph:
Hail Mary STN:
(C) Hike [0,5]
(20 yardline) ( Touchdown ! )
(QB) Pass to Endzone [5,8]
(R) Run to Endzone [7,9]
(R) Catch Football [0,2]
18
1.) Arc Change Without Effect to Shortest-Path
Rule 1: If an arc change doesn’t affect the shortest-path, then no update is needed.
0
14
5
155
5
13
13
13
5
0
0
0
00
8
-5
9
-7
0
00
0
2
0
Hail Mary STN:
(C) Hike [0,5]
(20 yardline) ( Touchdown ! )
(QB) Pass to Endzone [5,8]
(R) Run to Endzone [7,9]
(R) Catch Football [0,2]
(R) Run to Endzone [7,11]
11
Hail Mary Distance Graph:
19
2.) Shortest-Path Improvement
Rule 2: If an arc change improves shortest-path, then just propagate the improved costs
0
14
5
155
5
13
13
13
5
0
0
0
00
8
-5
9
-7
0
00
0
2
0
Hail Mary STN:
(C) Hike [0,5]
(20 yardline) ( Touchdown ! )
(QB) Pass to Endzone [5,8]
(R) Run to Endzone [7,9]
(R) Catch Football [0,2]
(QB) Pass to Endzone [5,7]
7
12
1412
12
Hail Mary Distance Graph:
20
3.) Invalidation of Shortest-Path
Rule 3: If an arc change invalidates shortest-path:
First, reset all invalidated nodes to infinity, and then propagate improved costs from the changed arc’s head node.
0
14
5
155
5
13
13
13
5
0
0
0
00
8
-5
9
-7
0
00
0
2
0
Hail Mary STN:
(C) Hike [0,5]
(20 yardline) ( Touchdown ! )
(QB) Pass to Endzone [5,8]
(R) Run to Endzone [7,9]
(R) Catch Football [0,2]
(QB) Pass to Endzone [5,9]
9Hail Mary Distance Graph:
14
14
14
16
21
Glass Box Behavior of ITC
Rule 1: No effect to shortest path
Rule 2: Shortest path improvement
Rule 3: Invalidation of shortest path
START END
22
Experimental Results
The Incremental Update Algorithm (ITC) was tested on:
- UAV Scenarios - Randomly Generated STNs
NFZ1
NFZ2
WaterA2
WaterA1
WaterB1
WaterB2
Fire1
Fire2
Seeker UAV
Water UAV
No-Fly Zone
Legend:
Fire
Water
UAV Base
UAV Base
Plan Goal: Extinguish All FiresVehicles: Two Seeker UAVs
One Water UAV Resources: Fuel & Water
23
Experimental Results: UAV Scenarios
Comparison of Algorithm Runtime
0
2
4
6
8
10
12
1 10 19 28 37 46 55 64 73 82 91
Number of Activities
Alg
orith
m R
untim
e (s
ec)
Non-incrementalAlgorithm
IncrementalAlgorithm
Comparison of Algorithm Runtime
0
2
4
6
8
10
12
1 10 19 28 37 46 55 64 73 82 91
Number of Activities
Alg
orith
m R
untim
e (s
ec)
Non-incrementalAlgorithm
IncrementalAlgorithm
24
Experimental Results: Random STNs
Planner Performance Tests(on 100 randomly generated TPNs)
0
50
100
150
200
250
300
350
400
0 20 40 60 80 100 120
Th
ou
san
ds
# of Choice Nodes
# o
f T
emp
ora
l Co
nsi
sten
cy C
hec
ks inChronological BTw ithout ITC
Chrono BT w ith ITC
25
Key Assumptions and Limitations
•Kirk’s pre-defined operators are sufficiently expressive - Desired plan is expressible with Kirk’s pre-defined operators
•Activities are controllable- dispatcher can pick any specific time between the lower bound and upper bound of an activity (e.g. Hike [0,5] )
• There is no innovative problem solving or logical deduction when planning fails
26
Conclusions
• Automated Planning is Important to Aerospace Engineering
• Kirk allows flexibility in planning through flexible time-bounds
•With Incremental Update Rules, Kirk can utilize previous computations
• Experimental Data shows an order of magnitude improvement
May 12, 2004
Massachusetts Institute of Technology
Questions ?
May 12, 2004
Massachusetts Institute of Technology
Additional Slides
29
Origins of Kirk
• Deep Space 1 mission– NASA’s first fully autonomous spacecraft demonstration
• Used the HSTS planner– Used flexible time (Temporal Constraint Graph)
• Developed into Europa and then Kirk
30
Hail Mary: ( sequence ( C: hike ) ( parallel ( R: Run to Endzone ) ( QB: Throw to Endzone ) ) ( R: Catch Football ) )
Another Key Idea: Reducing Complexity
Kirk restricts activity plans to predefined operators.
Touchdown !
20 yardline (C) Hike(R) Run to Endzone
(R) Catch Football(QB) Pass to Endzone
Another Representation of the Hail Mary Activity Plan:
20
10
G
QB
C R
Hail Mary:
31
Using Conflicts to Guide the Search
• A Simple Temporal Network formed from the pre-defined operators can be transformed into a Conditional CSP.
• Then, Conflict-Directed Search Algorithms from the CSP literature such as Dynamic Backtracking and Conflict- Directed A* can be used to guide the search.
• Sometimes the conflicts returned by ITC aren’t focused enough:Planner Performance Tests
(on 100 randomly generated TPNs)
0
50
100
150
200
250
300
350
400
0 20 40 60 80 100 120
Th
ou
san
ds
# of Choice Nodes# o
f T
emp
ora
l Co
nsi
sten
cy C
hec
ks
in
Chronological BTw ithout ITC
Chrono BT w ith ITC
Dynamic BT w ith ITC
~ 3 sec
~ 0.5sec
Structured Test to show the Advantage of Dynamic Bactracking with ITC
0
20
40
60
80
100
120
2 4 6 8 10 12 14
Th
ou
sa
nd
s
# of parallel decision nodes
# o
f te
mp
ora
l co
nsi
sten
cy c
hec
ks in
ChronologicalBT w ithout ITC
Chrono BTw ith ITC
Dynamic BTw ith ITC
32
ITC Pseudo-codevoid Initialize(){01} Q := {02} for all sV(G) {03} d(s) = ∞;{04} p(s) = unknown;{05} d(sstart) = 0;{06} Q.Insert(sstart);
Conflict CheckTemporalConsistency(G){07} while !Q.empty(){08} u = Q.pop(){09} for v Succ(u){10} dval = Update(u,v){11} if(dval) < 0 {12} c = CompletedCycle(v);{13} if(c){14} return ExtractConf(c, );{15} return 0;
value Update(p,x){16} if (d(y) > d(x) + c(x,y)){17} d(y) := d(x) + c(x,y);{18} p(y) := x;{19} Q.Insert(y);{20} return d(y);
NodeCompletedCycle(v){21} if L.contains(v){22} return v;{23} else{24} L.add(v){25} return 0;
voidModifyConstraint(x,y,l,u){26} ModifyArc(arc(y,x),-l){27} ModifyArc(arc(x,y),u)
ConflictExtractConflict(c,l){28} if l.contains(c){29} return l;{30} else{31} l.add{c};{32} ExtractConflict(p(c),l);
voidModifyArc(arc,c){33} setCost(arc,c);{34} if (d(arc.head > d(arc.tail) + c){35} d(arc.head) := d(arc.tail) + c;{36} p(arc.head) := arc.tail; {37} Insert(arc.head);{38} elseif (d(arc.head) < d(arc.tail) + c)
AND (p(arc.head) == arc.tail)){39} d(arc.head) := ∞;{40} p(arc.head) := unknown;{41} InvalidateSupports(arc.head);{42} InsertParents(arc.head);
void InsertParents(n){43} for all m Pred(n){44} Insert(m);{45} if(p(m) == n){46} if(m == sstart){47} d(m) := 0; {48} else {49} d(m) := ∞;{50} p(m) := unknown;{51} InsertParents(m);
voidInvalidateSupports(n){52} for s Succ(n){53} if(p(s) == n){54} d(n) := ∞;{55} p(n) := unknown;{56} InsertParents(s);{57} InvalidateSupports(s);
Figure 6 ITC Pseudo-Code
33
ITC Arc Update Rules (Pseudocode)
{01} if (d(arc.head) < d(arc.tail) + c) AND{02} p(arc.head) == arc.tail{03} d(arc.head) := ∞;{04} p(arc.head) := unknown{05} InvalidateSupports(arc.head);{06} InsertParents(arc.head);
{01} if (d(arc.head) > d(arc.tail) + c){02} d(arc.head) := d(arc.tail) + c;{03} p(arc.head) := arc.tail; {04} Insert(arc.head);
{01} if (d(arc.head) < d(arc.tail) + c) ANDp(arc.head) ≠ arc.tail)
{02} return;
Three Arc Update Rules:
1.) Arc Change With No Effect to Shortest-Path
2.) Arc Change Improves Shortest-Path
3.) Arc Change Invalidates Shortest-Path
34
References
• R. Ahuja, T. Magnanti, J. Orlin. Network Flows: Theory, Algorithms, and Applications. Prentice Hall, 1993.• R. Bellman. On a Routing Problem. Quaterly of Applied Mathematics, vol. 16, pp. 87 90, 1958.• N. Chandrachoodan, S. Bhattachryya, and K.J. Liu. Adaptive Negative Cycle Detection in Dynamic Graphs. In Proc. of
the International Symposium on Circuits and Systems, May 2001. • R. Dechter, I. Meiri, J. Pearl. Temporal Constraint Networks. Artificial Intelligence, 49:61-95, May 1991. • T. Estlin, G. Rabideau, D. Mutz, S. Chien. Using Continuous Planning Techniques to Coordinate Multiple Rovers.
Electronic Transactions on Artificial Inttligence, 4:45-57, 2000.• P. Kim, B. Williams, and M. Abrahmson. Executing Reactive, Model-based Programs through Graph-based Temporal
Planning. In Proceedings of IJCAI-2001, Seattle, WA, 2001.• S. Koenig and M. Likhachev. Incremental A*. In Adv. in Neural Information Processing Systems 14, 2001.• D. McAllester. Truth Maintenance. In Proceedings of AAAI-90, 1990, 1109-1116.• G. Rabideau, R. Knight, S. Chien, A. Fukunaga, A. Govindjee. Iterative Repair Planning for Spacecraft Operations in
the ASPEN System. ISAIRAS, Noordwijk, The Netherlands, June 1999.• Tsmardinos, N. Muscettola, and P.Morris. Fast transformation of temporal plans for efficient execution. In AAAI-98,
1998.• B.C. Williams and R.J. Ragno. Conflict-directed A* and its role in model-based embedded systems. Journal of Discrete
Applied Math, • A. Stentz. Optimal and efficient path planning for partially known environments. In Proceedings of IEEE ICRA, May
1994.• N. Muscettola, P. Morris, B. Pell, and B. Smith. Issues in temporal reasoning for autonomous control systems. In
Autonomous Agents, 1998.• G. Verfaillie and T. Schiex. Dynamic backtracking for dynamic constraint satisfaction problems. In Proceedings of the
ECAI'94 Workshop on Constraint Satisfaction Issues Raised by Practical Applications, Amsterdam, The Netherlands, pages 1-8, 1994.
• Matthew L. Ginsberg and David A. McAllester. Gsat and dynamic backtracking. In Principles of Knowledge Representation and Reasoning: Proceedings of the Fourth International Conference (KR `94), 1994.
• P. Prosser, Hybrid algorithms for the constraint satisfaction problem, Computational Intelligence 3 (1993) 268—299.