The Role of Domain-Specific Knowledge in the

Planning as SatisfiabilityFramework

The Role of Domain-Specific Knowledge in the

Planning as SatisfiabilityFramework

Henry KautzAT&T Labs

Bart SelmanCornell University

2

A Core Computation ProblemA Core Computation Problem

Focus: Classical state-space planning, extended with parallel actions

Core problem - similar computational issues arise in many other models

• Reactive plans

• Planning with uncertainty and utilities

• Continuous processes

• Metric time

Many results directly extend to richer formalisms

3

State-space PlanningState-space Planning

Find a sequence of operators that transform an initial state to a goal state

State = complete truth assignment to a set of variables (fluents)

Goal = partial truth assignment (set of states)

Operator = a partial function State State

• specified by three sets of variables:precondition, add list, delete

list

4

ParallelismParallelism

Operators may be applied in parallel when all orderings are well defined and equivalent

(Op1 || Op2)(s) = Op2(Op1(s)) = Op1(Op2(s))

A special form of non-linear plans

• Only allows parallel actions, not parallel action sequences

• Easy to serialize

5

Abdundance of Negative Complexity Results

Abdundance of Negative Complexity Results

I. Domain-independent planning: PSPACE-complete or worse

(Chapman 1987; Bylander 1991; Backstrom 1993)

II. Domain-dependent planning: NP-complete or worse

(Chenoweth 1991; Gupta and Nau 1992)

III. Approximate planning: NP-complete or worse

(Selman 1994)

6

PracticePractice

Domain-independent planning systems can only generate relatively short plans.

Practical systems minimize or eliminate search by employing

• Search control rules, hand-tailored to the search engine and the particular search space(Sacerdoti 1975, Slaney 1996, Bacchus 1996)

• Pre-compiling entire state-space to a reactive finite-state machine (Agre & Chapman 1997, Williams & Nayak 1997)

Scaling remains problematic when state space is large or not well understood!

7

Planning as InferencePlanning as Inference

• Planning as first-order theorem proving (Green 1969)

computationally infeasible

• STRIPS (Fikes & Nilsson 1971)

very hard

• Partial-order planning (modal truth criteria) (Tate 1977, Chapman 1985, McAllester 1991, Smith & Peot 1993)

can be more efficient, but still hard (Minton, Bresina, & Drummond 1994)

• SATPLAN: planning as propositional reasoning

8

ApproachApproach

SAT encodings are designed so that plans correspond to satisfying assignments

Use recent efficient satisfiability procedures (systematic and stochastic) to solve

Evaluation performance on benchmark instances

9

SATPLANSATPLAN

axiomschemas instantiated

propositionalclauses

satisfyingmodelplan

mapping

length

problemdescription

SATengine(s)

instantiate

interpret

(New implementation blackbox takes STRIPS input - these experiments based on older hand-created “state-based” encodings)

10

SAT EncodingsSAT Encodings

Propositional CNF: no variables or quantifiers

Sets of clauses specified by axiom schemas

• fully instantiated before problem-solving

Discrete time, modeled by integers

• state predicates: indexed by time at which they hold

• action predicates: indexed by time at which action begins– each action takes 1 time step

– many actions may occur at the same step

fly(Plane, City1, City2, i) at(Plane, City2, i +1)

11

Solution to a Planning ProblemSolution to a Planning Problem

A solution is specified by any model (satisfying truth assignment) of the conjunction of the axioms describing the initial state, goal state, and operators

Easy to convert back to a STRIPS-style plan

12

Satisfiability Testing ProceduresSatisfiability Testing Procedures

Systematic, complete procedures• Depth-first backtrack search

(Davis, Putnam, & Loveland 1961)

– unit propagation, shortest clause heuristic

• One of the best implementations: ntab (Crawford & Auton 1997)

– see also: satz (Li), CSAT (Dubois), MODOC (van Gelder)

Stochastic, incomplete procedures• Inspired by Mins-Conflict algorithm

(Adorf & Johnson 1990; Minton, Johnson, Philips, & Laird 1992)

• GSAT (Selman et. al 1993)

• Current fastest: Walksat (Selman & Kautz 1993)

– greedy local search + noise to escape local minima

13

Planning Benchmark Test SetPlanning Benchmark Test Set

Extension of Graphplan test set

blocks world - up to 18 blocks, 1019 states

logistics - complex, highly-parallel transportation domain.

Logistics.d:

• 2,165 possible actions per time slot

• 1016 legal configurations (22000 states)

• optimal solution contains 74 distinct actions over 14 time slots

Problems of this size never previously handled by state-space planning systems

14

Solution of Logistics ProblemsSolution of Logistics Problems

0.01

0.1

1

10

100

1000

10000

100000

rocket.a rocket.b log.a log.b log.c log.d

log

so

luti

on

tim

e

Graphplan

ntab/STRIPS

w alksat/STRIPS

ntab/STATE

w alksat/STATE

15

Why SATPLAN WorksWhy SATPLAN Works

More flexible than forward or backward chaining

• Systematic: most unit propagation at most highly constrained states

• Stochastic: iterative repair

Space for time tradeoff

• Less overhead since does not have to instantiate variable during search

Randomized algorithms less likely to get trapped along bad paths

16

Domain Dependent PlanningDomain Dependent Planning

Efficient representations and (randomized) SAT engines extend the range of domain-independent planning

Ways to further improvement:

• Better general search algorithms

• Incorporate (more) domain dependent knowledge

17

Kinds of KnowledgeKinds of Knowledge

* About domain itself• a truck is only in one location

• airplanes are always at some airport

* About good plans• do not remove a package from its destination location

• do not unload a package and immediate load it again

X About how to search• plan air routes before land routes

• work on hardest goals first

18

Expressing KnowledgeExpressing Knowledge

Such information is traditionally incorporated in the planning algorithm itself

– or in a special programming language

Instead: we propose expressing as additional declarative axioms

• Problem instance: operator axioms + initial and goal axioms + heuristic axioms

• Domain knowledge constraints on search and solution spaces

• Independent of any search engine strategy

NOT logic programming!

19

Logical Status of HeuristicsLogical Status of Heuristics

1. Entailed by operator axioms: conflicts and derived effects

fly(plane,d1,i) and fly(plane,d2,i) conflict

2. Entailed by operators + initial state axioms: state invariants

a truck is at only one location

3. Entailed by operators + initial + goal + length: optimality conditions

do not return a package to a location

4. New constraints on problem instance: simplifying assumptions

Once a truck is loaded, it should immediately move

20

Axiomatic FormAxiomatic Form

Invariant: A truck is at only one location

at(truck,loc1,i) & loc1 loc2 at(truck,loc2,i)

Optimality: Do not return a package to a

location

at(pkg,loc,i) & at(pkg,loc,i+1) & i<j at(pkg,loc,j)

Simplifying: Once a truck is loaded, it should immediately move

in(pkg,truck,i) & in(pkg,truck,i+1) &at(truck,loc,i+1)

at(truck,loc,i+2)

21

QuestionsQuestions

Does it work?

• Additional axioms might just blow up instance with redundant information

Is effect independent of search engine?

• Simplifying assumptions might hurt stochastic methods by reducing solution density

Can we predict the most useful level of heuristic axioms?

• What is relation of difficulty to problem size?

22

Experiment 1: Blocks WorldExperiment 1: Blocks World

Observation: Original BW examples for SATPLAN included some state invariant axioms

• After simplification by unit propagation, invariant axioms make instance smaller and easier to solve (Ernst, Millstein, Weld 1997)

Experiment: 3 versions of each instance• minimal set of axioms• + state invariants• + optimality (“don’t move a block twice in a row”)

vars walksat ntabbw.b.ops 1635 13.08 5.13bw.b.state 1865 2.22 0.66bw.b.heur 1876 0.75 0.38bw.d.ops 8282 >48 hr >48 hrbw.d.state 9023 688 >48 hrbw.d.heur 9042 643 2651

23

Experiment 2: LogisticsExperiment 2: Logistics

h1: Optimality conditions• Once a package leaves a location, it never returns

h2, h3: Simplifying assumptions• A package is never in any city other than its origin

or destination cities– rules out solutions where packages are transferred

between airplanes in an intermediate city

• Once a vehicle is loaded, it should immediately move– rules out solutions where vehicles are loaded

incrementally

h4: More optimality conditions• A package never leaves its destination city

24

logistics size in variableslogistics size in variables

500

600

700

800

900

1000

1100

1200

none h1 h2 h3 h4

nu

mb

er o

f va

rial

bes

log.a

log.b

log.c

25

logistics size in clauseslogistics size in clauses

4000

6000

8000

10000

12000

14000

16000

none h1 h2 h3 h4

nu

mb

er o

f c

lau

ses

log.a

log.b

log.c

26

walksat solution of logisticswalksat solution of logistics

0

1

2

3

4

5

6

7

8

none h1 h2 h3 h4

no

rmal

ized

so

luti

on

tim

e

log.a

log.b

log.c

log.d

27

ntab solution of logisticsntab solution of logistics

0.1

1

10

100

1000

10000

100000

none h1 h2 h3 h4

log

no

rmal

ized

so

luti

on

tim

e

log.a

log.b

log.c

28

AnswersAnswers

Does it work?

• YES, 10 to 100+ times speedup

Is effect independent of search engine?

• YES, same heuristics best for systematic and stochastic engines - but needs more investigation

Can we predict the most useful level of heuristic axioms?

• USUALLY point at which problem size is minimized after simplification by unit propagation (40% - 70% reduction)

• BUT larger simplified instances still easier than “no heuristic” case

29

How to Generate HeuristicsHow to Generate Heuristics

Introspection• Try to capture “obvious” inferences that are hard to

deduce (McAllester)

Compilation• Fix operators and initial or goal state, generate

tractable equivalent theory (Kautz & Selman)

• In worse case, tractable theory is exponentially large but can be efficient in practice

EBL (Minton, Kambhampati)

Static analysis (Smith, Etzioni, Knoblock, Peot)

• How to adapt to generate constraints on plans instead of explicit control rules?

30

SummarySummary

Easy to encode domain-specific knowledge in the planning as satisfiablity frame

• Key to next order-of-magnitude scaling

Purely declarative assertions• Not logic programming!

Makes clear logical status of different heuristics• invariants, optimality, simplifying

• entailed admissible

Heuristics are independent from the SAT engine• Can use same axioms for radically different

problem solvers

31

Summary (continued)Summary (continued)

Dramatic speedup possible

• Stochastic: speedup 6X - 10X

• Systematic: speedup 100X +

For largest, hardest problems, additional domain knowledge allows systematic solvers to become competitive

• but in some domains stochastic solvers still show better scaling

32

Logistics Best Solution TimesLogistics Best Solution Times

0.01

0.1

1

10

100

1000

10000

100000

log.a log.b log.c log.d

log

so

luti

on

tim

e

w alksat

ntab

satz

33

Current & Future WorkCurrent & Future Work

Automatic generation of axioms from STRIPS input• efficient interleaving instantiation and pruning

(Kambhampati 1997)

Randomized systematic search

• broad coverage across domains• robust against instances that are hard for

deterministic strategies (“heavy tails”) (Gomes, Selman, Kautz 1997)

blackbox• adapts graphplan for generating wffs

• randomized version of satz (Li 1997)

• now solves log.d in 1 minute directly from STRIPS input, without hand-encodings or domain knowledge!

34

blackbox version 9Bcommand line: blackbox -o logistics.pddl -f logistics_prob_d_len.pddl -solver compact -l -then satz -cutoff 25 -restart 10

----------------------------------------------------Converting graph to wff6151 variables243652 clausesInvoking simplifier compactVariables undetermined: 4633Non-unary clauses output: 139866----------------------------------------------------Invoking solver satz version satz-rand-2.1Wff loaded[1] begin restart[1] reached cutoff 25 --- back to root[2] begin restart[2] reached cutoff 25 --- back to root [3] begin restart[3] reached cutoff 25 --- back to root [4] begin restart[4] reached cutoff 25 --- back to root [5] begin restart**** the instance is satisfiable ********* verification of solution is OK ****

total elapsed seconds = 25.930000----------------------------------------------------Begin plan1 drive-truck_ny-truck_ny-central_ny-po_ny

35

Begin plan1 drive-truck_ny-truck_ny-central_ny-po_ny1 drive-truck_sf-truck_sf-airport_sf-po_sf1 load-truck_package5_bos-truck_bos-po1 drive-truck_pgh-truck_pgh-airport_pgh-central_pgh1 fly-airplane_airplane2_pgh-airport_sf-airport1 load-truck_package6_bos-truck_bos-po2 load-truck_package2_pgh-truck_pgh-central2 load-truck_package4_ny-truck_ny-po2 load-truck_package7_ny-truck_ny-po2 load-truck_package3_pgh-truck_pgh-central2 drive-truck_bos-truck_bos-po_bos-airport_bos2 load-airplane_package8_airplane2_sf-airport2 fly-airplane_airplane1_pgh-airport_sf-airport2 drive-truck_la-truck_la-po_la-airport_la3 fly-airplane_airplane2_sf-airport_bos-airport3 unload-truck_package6_bos-truck_bos-airport3 drive-truck_pgh-truck_pgh-central_pgh-airport_pgh3 fly-airplane_airplane1_sf-airport_pgh-airport3 unload-truck_package5_bos-truck_bos-airport3 drive-truck_ny-truck_ny-po_ny-airport_ny3 drive-truck_sf-truck_sf-po_sf-airport_sf4 unload-truck_package3_pgh-truck_pgh-airport4 unload-truck_package2_pgh-truck_pgh-airport4 unload-truck_package4_ny-truck_ny-airport4 load-airplane_package6_airplane2_bos-airport4 load-airplane_package5_airplane2_bos-airport4 drive-truck_la-truck_la-airport_la-po_la4 drive-truck_bos-truck_bos-airport_bos-central_bos4 unload-truck_package7_ny-truck_ny-airport5 drive-truck_ny-truck_ny-airport_ny-po_ny5 drive-truck_bos-truck_bos-central_bos-po_bos5 load-airplane_package2_airplane1_pgh-airport5 drive-truck_la-truck_la-po_la-central_la5 drive-truck_pgh-truck_pgh-airport_pgh-po_pgh5 load-airplane_package3_airplane1_pgh-airport5 fly-airplane_airplane2_bos-airport_ny-airport6 drive-truck_sf-truck_sf-airport_sf-central_sf6 unload-airplane_package6_airplane2_ny-airport6 load-airplane_package4_airplane2_ny-airport6 drive-truck_la-truck_la-central_la-po_la6 drive-truck_bos-truck_bos-po_bos-airport_bos6 load-airplane_package7_airplane2_ny-airport6 drive-truck_ny-truck_ny-po_ny-airport_ny6 unload-airplane_package8_airplane2_ny-airport6 fly-airplane_airplane1_pgh-airport_sf-airport6 load-truck_package1_pgh-truck_pgh-po7 fly-airplane_airplane2_ny-airport_la-airport7 fly-airplane_airplane1_sf-airport_bos-airport7 load-truck_package9_sf-truck_sf-central7 load-truck_package6_ny-truck_ny-airport

7 drive-truck_bos-truck_bos-airport_bos-central_bos7 drive-truck_pgh-truck_pgh-po_pgh-airport_pgh7 load-truck_package8_ny-truck_ny-airport8 drive-truck_sf-truck_sf-central_sf-po_sf8 fly-airplane_airplane2_la-airport_pgh-airport8 unload-truck_package1_pgh-truck_pgh-airport8 drive-truck_bos-truck_bos-central_bos-po_bos8 drive-truck_ny-truck_ny-airport_ny-central_ny8 fly-airplane_airplane1_bos-airport_la-airport8 drive-truck_la-truck_la-po_la-airport_la9 unload-airplane_package7_airplane2_pgh-airport9 unload-truck_package8_ny-truck_ny-central9 unload-airplane_package5_airplane2_pgh-airport9 unload-truck_package9_sf-truck_sf-po9 unload-airplane_package3_airplane1_la-airport9 unload-truck_package6_ny-truck_ny-central9 drive-truck_pgh-truck_pgh-airport_pgh-po_pgh9 load-airplane_package1_airplane2_pgh-airport10 drive-truck_ny-truck_ny-central_ny-po_ny10 fly-airplane_airplane2_pgh-airport_bos-airport10 load-truck_package3_la-truck_la-airport10 fly-airplane_airplane1_la-airport_ny-airport10 drive-truck_pgh-truck_pgh-po_pgh-airport_pgh11 drive-truck_bos-truck_bos-po_bos-airport_bos11 drive-truck_ny-truck_ny-po_ny-airport_ny11 unload-airplane_package2_airplane1_ny-airport11 drive-truck_la-truck_la-airport_la-central_la11 drive-truck_sf-truck_sf-po_sf-airport_sf11 unload-airplane_package1_airplane2_bos-airport11 load-truck_package7_pgh-truck_pgh-airport11 load-truck_package5_pgh-truck_pgh-airport12 drive-truck_sf-truck_sf-airport_sf-po_sf12 load-truck_package1_bos-truck_bos-airport12 fly-airplane_airplane2_bos-airport_la-airport12 load-truck_package2_ny-truck_ny-airport12 fly-airplane_airplane1_ny-airport_pgh-airport12 drive-truck_pgh-truck_pgh-airport_pgh-po_pgh12 unload-truck_package3_la-truck_la-central13 drive-truck_ny-truck_ny-airport_ny-po_ny13 load-truck_package3_la-truck_la-central13 load-truck_package9_sf-truck_sf-po13 drive-truck_bos-truck_bos-airport_bos-po_bos13 unload-truck_package5_pgh-truck_pgh-po13 unload-airplane_package4_airplane2_la-airport14 unload-truck_package9_sf-truck_sf-po14 unload-truck_package1_bos-truck_bos-po14 unload-truck_package7_pgh-truck_pgh-po14 unload-truck_package2_ny-truck_ny-po14 unload-truck_package3_la-truck_la-centralEnd plan

36

Current & Future Work (continued)

Current & Future Work (continued)

Future extensions to blackbox:

• Specification of heuristic axioms in the STRIPS-style input– state invariants - straightforward

– optimality conditions - temporal logic?

• Automatic generation of heuristic axioms

DOWNLOAD latest version from

www.research.att.com/~kautz/blackbox

37

What is Planning?What is Planning?

Informally: find a (partially) ordered set of actions that transform a given initial state to a specified goal state.

• in most general case, can cover most forms of problem solving

• special case of program synthesis

• scheduling: fixes set of actions, need to find optimal total ordering

- scheduling problems often (near) linear, may be solvable by linear relaxation techniques

- planning problems typically highly non-linear, require combinatorial search

38

Some Applications of PlanningSome Applications of Planning

Autonomous systems

Many NASA applications:

• Deep Space One Remote Agent

• Long-range mission planning

• Communication planning & scheduling

Softbots - software robots

• Internet agents, program assistants

• AI “characters” in games, entertainment

User modeling / discourse

Synthesis, diagnosis, experiment design, bug-finding (goal = undesirable state), ...

39

Walksat ProcedureWalksat Procedure

Start with random initial assignment.

Pick a random unsatisfied clause.

Select and flip a variable from that clause:

• With probability p, pick a random variable.

• With probability 1-p, pick greedily– a variable that minimizes the number of unsatisfied

clauses

Repeat to predefined maximum number flips; if no solution found, restart.