Efficient Automated Planning with New Formulations

Ruoyun HuangWashington University in St. Louis

Classical AI Planning (STRIPS)loc1 loc2

Init State: (AT pkg loc1); (AT truck loc2);

loc1 loc2

Goal: (AT pkg loc2)

Applicable actions: MOVE; LOAD;UNLOAD;

?

One solution with time step N=4: Time Step 1: MOVE truck loc2 loc1 Time Step 2: LOAD pkg truck loc1Time Step 3: MOVE truck loc1 loc2 Time Step 4: UNLOAD pkg truck loc2

2

Mars rover by NASA High speed printer by PARC Hubble space telescope Mobile robots Anti-air defense system Airport scheduling Biological network planning Natural language processing …

Challenges High complexity Expressiveness

Applications and Challenges

3

Formulation/Representation is vital to problem solving

Different formulations of planning STRIPS representation[Fikes:71]

SAS+ representation[Backstrom:96]

In planning by convert-and-solve methods Convert planning problem into Satisfiability (SAT)

Constraint Satisfaction Programming (CSP), or Integer Programming

How we formulate is the key

Motivation

4

SAT

OutlineAutomated Planning

Classical Planning

STRIPS

Search

SAS+

SAT Search SAT12 3

Blue backgrounds indicate my/our workSAT = SatisfiabilityPOR = Partial Order Reduction

Temporal Planning

Temporally Expressive

Applications

Web Service Composition

Parallel Algorithms

POR Theory

SAS+ Representationloc1 loc2

in truck

at loc1 at loc2

V(pkg)

at loc1 at loc2

V(truck)

AT pkg loc1 AT pkg loc2 IN pkg truck AT truck loc1 AT truck loc2

Strips SAS+

Transition: Change between values in a multi-valued variable

Transitionspkg:loc1truckpkg:truckloc1pkg:truckloc2pkg:loc2truck

6

DTG(Domain Transition Graph)

SAS+ is more compactfact…fact

… fact…fact

…

fact…

fact…

fact…

fact…fact

…

fact…

fact…

…… 2F…

… …

…… …

… ……

10F/10

F: number of STRIPS factsAssume avg. facts/value per DTG is 10

fact… fact

…

7

SAS+ also captures more hidden problem structures

Heuristic Function Inadmissible heuristic function[Helmert:06]

Admissible heuristic function [Helmert:08,Katz:08]

Exception: Mixed integer programming[Briel:04]

Related Works on SAS+

8 call for more comprehensive studies

Planning models & algorithms

Planning as Search

Heuristic function design

Outline

SAS+ Based Search Model[Chen, Huang, Zhang: 08]

Automated Planning

Classical Planning

STRIPS

SAS+

Search SAT Search SAT1

Temporal

Applications

A popular technique for planning

Limitation of heuristic search Exponential number of states to visit[Helmert:08]

Our contribution: Search in abstraction state spaces

Smaller space Comparing with HTN planning

We don need domain knowledge

Heuristic Search

10

11

Causal Dependency and Causal Graphloc1 loc2

at loc1 at loc2

in truck

at loc1 at loc2

pkg’s DTG

truck’s DTG

pkg’s DTG

truck’s DTG

LOAD pkg truck loc2:Pre: (AT truck loc2)

Causal Graph[Helmert08]

12

Motivation of Abstraction Causal graph induces hierarchies within planning

problemsTo build a house

1. Drive to the city hall2. Fill out an application3. Drive to the bank4. Ask for money5. Drive somewhere6. Look for a builder7. Sign a Contract8. Build foundation9. Build frame10. Build roof11. Build wall13. Buy furniture … …

1. Get a permit2. Get enough money3. Construction4. Interior … …

1.a Drive to the city call1.b Fill out an application

2.a Drive to bank2.b Fill out an application

3.a Find a builder3.b Sign a contract3.c Build foundation3.d Build frame3.e Build roof3.f build wall

… …

13

Abstraction State Space

truck’s DTG

package’s DTG

full state space

Abstraction

loc1 loc2

1. LOAD pkg truck loc1 2. UNLOAD pkg truck loc2

1.a MOVE truck loc2 loc11.b LOAD pkg truck loc1

2.a MOVE truck loc1 loc22.b UNLOAD pkg truck loc2in truck

at loc1 at loc2 DTGs guide us to determine high-level plans

14

Abstraction State Space Search (Iteration #1)

… … … …

… … … …

Goal1Goal

2

Abstraction State

SpaceDTG DTG

DTG DTG DTG DTG

DTG DTG DTG DTG

Finds a plan quickly

15

Abstraction State Space Search (Iteration #2)

… … … …

… … … …

G1 G2

Abstraction State SpaceDTG DTG

DTG DTG DTG DTG

DTG DTG DTG DTG

Keep running to improve the plan quality

ResultsInstances Avg. Time Avg. Plan Length

DTG-Plan FastDownward DTG-Plan Fast

Downward DTG-Plan FastDownward

Openstack 28 28 3.39 10.91 131.6 131.5

Pathways 30 30 1.69 2.90 166.3 132.8Rovers 40 40 12.05 9.69 151.9 112.2Storage 17 18 0.58 1.78 15.9 14.6

TPP 30 30 9.16 47.16 172.2 133.1Trucks 16 10 19.19 53.04 31.50 28.4 The average running time and plan length are calculated by the

instances solved by both algorithms. Here DTG-Plan runs for a single iteration, more iterations will

improve the solution quality.

Results (Rovers Domain) Time

FastDownward Quality

Quality

Iter#1 Iter#2 Iter#3 Iter#1 Iter#2 Iter#3Rover3 0.09 0.212 0.209 15 18 13 13Rover4 0.09 0.218 0.209 8 13 8 8Rover5 0.11 0.308 2.236 33 27 27 22Rover6 0.14 0.707 15.896 38 46 46 36Rover7 0.13 0.37 0.428 23 36 20 18Rover8 0.23 0.656 9.98 26 36 31 29Rover9 0.23 0.824 19.068 36 54 42 40Rover10 0.25 1.523 55.258 37 41 41 38Rover11 0.28 0.973 4.421 56 71 46 36Rover12 0.24 0.652 0.912 21 32 24 19Rover13 0.41 6.579 120.912 81 80 58 56Rover14 0.3 0.91 24.76 32 45 35 33Rover15 0.41 1.559 27.127 44 49 44 42Rover16 0.48 3.564 28.486 51 77 46 46Rover17 0.79 17.775 366.27 52 69 69 48Rover18 1.37 4.792 516.72 47 55 51 47

Downward has a similar running time to Iter#1

Outline

Enhanced mutual exclusion[Chen, Huang, Xing, Zhang: 09]

2

Automated Planning

Classical Planning

STRIPS

SAS+

Search SAT Search SAT

Temporal

Applications

19

Planning as SatisfiabilityGiven planning problem P:

Encode P with time N, resulting in SAT formula EN

Use a SAT solver to solve EN

EN is solvable?

N = 0

Decode and output solutionY

N

N = N + 1

Encoding is the key!

Strips Based Encoding

(at pkg loc1)

(at truck loc2)

move

(at pkg loc1)

(at truck loc2)

(at truck loc1)

Time step 1 Time step 2

(at pkg loc1)

move

move

(at truck loc2)

(at truck loc1)

load (in pkg truck) ……………

Time step 3,4,5,…

Compile from planning graph

20

SAT Instance:VariablesClauses

Planning GraphNodes (facts, actions)Edges

Results in a whole chunk of encoding

Two facts/actions cannot be true at the same time Fact mutual exclusion Action mutual exclusion

Mutual Exclusion (Mutex)

Move-truck-loc1-loc2

Move-truck-loc2-loc1

(in pkg truck)(at truck loc1) (at truck loc2)

(at pkg loc1) (at pkg loc2)

(in pkg truck)(at truck loc1) (at truck loc2)

(at pkg loc1) (at pkg loc2)

A key reason for the efficiency of SAT-based planning21

22

Mutual Exclusion

Time #2 Time #3 #4 #5 #6 #7 #8 #9 #10Time #1

Invalid

(at truck loc1) (at truck loc2)

23

Long Distance Mutual Exclusion(Londex)

Time #2 Time #3 #4 #5 #6 #7 #8 #9 #10Time#1

Invalid

(at truck loc1) (at truck loc4)

DTG of truck(at truck loc1) (at truck loc2) (at truck loc3) (at truck loc4)

Significant speed-up Hundreds times faster in some cases (compared to

SatPlan04)

Limitation Too large encoding size (ten times in average)

Results of Londex

24

Outline

A SAS+ based encoding scheme[Huang,Chen,Zhang:10]

AAAI’10 Outstanding Paper Award

3

Automated Planning

Classical Planning

STRIPS

SAS+

Search SAT Search SAT

Temporal

Applications

STRIPS versus SAS+STRIPS SAS+

Definition

a set of preconditions, a set of add effects, a set of delete effects

A set of transitions

Example

(LOAD pkg truck loc1)Pre:

(at truck loc1),(at pkg loc1) pkg:(loc1truck)

truck: (loc1loc1)Del: (at pkg loc1)Add: (in pkg truck)

Usually there are fewer transitions than actions

Hierarchical relationships between actions and transitions

Different Representations of Actions:

26

Overview of New Encoding

…………

SAT Instance (Part 1):transitions

SAT Instance (Part 2):matching actions and transitions (multiple independent ones)… … …

Transitions

Actions

t = 1 t =2 t =3

SAT Instance:Facts and actionsActions&

Facts

Planning graph

… … …

t = 1 t =2 t =3, 4, …

Strips Based Encoding

SAS+ Based New Encoding27

28

Clauses in New Encodingtruck:loc2

Time step 1 Time step 2

……………

Time step 3,4,5,…

pkg: loc1

truck:loc1 truck:loc1

truck:loc2

truck:loc1

truck:loc2

pkg: loc1pkg: truckpkg: loc2

pkg: loc1pkg: truckpkg: loc2

Find matchings

set of actions set of actions … …

pkg: truckpkg: loc2

1. Progression of transitions over time steps (blue one implies green ones)2. Initial state and goal (Bold ones)3. Matching actions and transitions4. Action mutual exclusions and transition mutual exclusions

State Space Size

…………

State Space (Part 1):O((2T)N)State Space (Part 2):O(NK)

… … …

Transitions

Actions

t = 1 t =2 t =3

State Space:O((2A)N)

ActionsPlanning graph

… … …

t = 1 t =2 t =3, 4, …

Strips Based Encoding

SAS+ Based New Encoding

29

Total: O((2T)NNK)

K K K

Usually there are fewer transitions than actions

A Number of actionsN Number of Time Step

T Number of transitions

Number of Solvable Instances

31SatPlan06L = SatPlan06 + Londex

Detailed ResultsSatPlan06 New Encoding

Instances Time (sec) #Variables #Clauses Size

(MB) Time #Variables #Clauses Size

Airport40 2239.4 327,515 13,206,595 807 583.3 396,212 3,339,914 208

Driverslog17 2164.8 61,915 2,752,787 183 544.1 74,680 812,312 56

Freecell4 364.3 17582 6,114,100 392 158.4 26,009 371,207 25

Openstack4 212.1 3,709 66,744 5 33.6 4,889 20,022 2

Pipesworld12 3147.3 30,078 13,562,157 854 543.7 43,528 634,873 44

TPP30 3589.7 97,155 7,431,062 462 1844.8 136,106 997,177 70

Trucks7 1076.0 21,745 396,581 27 245.7 35,065 255,020 18

Zeno14 728.4 26,201 6,632,923 421 58.7 17,459 315,719 18

32

Better performances in 10 benchmark domains out of 11 tested (from IPC3,4,5)

To planning The dynamic world constraints

To general artificial intelligence Bounded model checking[Clarke01]

Answer set programming[Lin02]

…

Significance of New Encoding

Compactness Problem structure

Future works Other (classical) planning algorithms, e.g. Partial

Order Causal Link, … More advanced planning models, e.g.

probabilistic, uncertainty, …

Conclusions on SAS+

Thank You