http://parasol.tamu.edu

Randomized Motion Planning: From Intelligent CAD to Computer

Animation to Protein Folding

Nancy M. Amato Parasol Lab,Texas A&M University

Motion Planning

start

goal obstacles

(Basic) Motion Planning:

Given a movable object and a description of the environment, find a sequence of valid configurations that moves it from the start to the goal.

The Alpha Puzzle

(dis)Assembly Puzzle

Hard Motion Planning Problems: Intelligent CAD Applications

Using Motion Planning to Test Design Requirements: •  Accessibility for servicing/assembly tested on physical “mock ups”. •  Digital testing saves time and money, is more accurate, enables more extensive testing, and is useful for training (VR or e-manuals).

Maintainability Problems: Mechanical Designs from GE

Hard Motion Planning Problems: Highly Articulated (Constrained) Systems

Hard Motion Planning Problems: Group Behaviors for multiple agents

Traversing a Narrow Passage

A “shepherd” herding a flock of ducks

People evacuating a building

Hard Motion Planning Problems: Deformable Objects

Find a path for a deformable object that can deform to avoid collision with obstacles •  deformable objects have infinite dof!

Hard Motion Planning Problems Computational Biology & Chemistry

•  Drug Design - molecule docking

•  Simulating Molecular Motions – study folding pathways & kinetics

Protein Folding

RNA Folding

Outline

•  C-space, Planning in C-space (basic definitions)

•  Probabilistic Roadmap Methods (PRMs) •  PRM variants (OBPRM, MAPRM) •  User-Input: Haptically Enhanced PRMs

•  PRMs for more `complex’ robots •  deformable objects •  group behaviors for multiple robots •  protein folding

Configuration Space (C-Space)

C-obst

C-obst

C-obst

C-obst

C-obst

C-Space

6D C-space (x,y,z,pitch,roll,yaw)

3D C-space (x,y,z) 3D C-space

(α,β,γ) α β γ

•  “robot” maps to a point in higher dimensional space •  parameter for each degree of freedom (dof) of robot •  C-space = set of all robot placements •  C-obstacle = infeasible robot placements

2n-D C-space (φ1, ψ1, φ2, ψ2, . . . , φ n, ψ n)

robot

obst

obst

obst

obst

x y

θ

C-obst

C-obst C-obst

C-obst

robot Path is swept volume

Motion Planning in C-space

Path is 1D curve

Workspace C-space Simple workspace obstacle transformed

Into complicated C-obstacle!!

Most motion planning problems of interest are PSPACE-hard [Reif 79, Hopcroft et al. 84 & 86]

The best deterministic algorithm known has running time that is exponential in the dimension of the robot’s C-space [Canny 86] •  C-space has high dimension - 6D for rigid body in 3-space •  simple obstacles have complex C-obstacles impractical to compute explicit representation of freespace for more than 4 or 5 dof

So … attention has turned to randomized algorithms which •  trade full completeness of the planner •  for probabilistic completeness and a major gain in efficiency

The Complexity of Motion Planning

1. Connect start and goal to roadmap

Query processing start

goal

Probabilistic Roadmap Methods (PRMs) [Kavraki, Svestka, Latombe,Overmars 1996]

C-obst

C-obst

C-obst

C-obst

Roadmap Construction (Pre-processing)

2. Connect pairs of nodes to form roadmap - simple, deterministic local planner (e.g., straightline) - discard paths that are invalid

1. Randomly generate robot configurations (nodes) - discard nodes that are invalid

C-obst

C-space

2. Find path in roadmap between start and goal - regenerate plans for edges in roadmap

PRMs: Pros & Cons

PRMs: The Good News 1. PRMs are probabilistically complete 2. PRMs apply easily to high-dimensional C-space 3. PRMs support fast queries w/ enough preprocessing

Many success stories where PRMs solve previously unsolved problems

C-obst

C-obst

C-obst

C-obst

C-obst

start

goal

PRMs: The Bad News

1. PRMs don’t work as well for some problems: –  unlikely to sample nodes in narrow passages –  hard to sample/connect nodes on constraint surfaces

Our work concentrates on improving PRM performance for such problems.

start

goal

C-obst

C-obst

C-obst

C-obst

Smarter Sampling Methods

Goal: Sample nodes in Narrow Passages

•  OBPRM: Obstacle-Based PRM •  sampling around obstacles

•  MAPRM: Medial-Axis PRM •  sampling on the medial axis

•  Repairing/Improving approximate paths •  transforming invalid samples into valid samples

OBPRM: An Obstacle-Based PRM [IEEE ICRA’96, IEEE ICRA’98, WAFR’98]

start

goal

C-obst

C-obst

C-obst

C-obst

To Navigate Narrow Passages we must sample in them •  most PRM nodes are where planning is easy (not needed)

PRM Roadmap

start

goal

C-obst

C-obst

C-obst

C-obst

Idea: Can we sample nodes near C-obstacle surfaces? •  we cannot explicitly construct the C-obstacles... •  we do have models of the (workspace) obstacles...

OBPRM Roadmap

1

3

2

4 5

OBPRM: Finding Points on C-obstacles

Basic Idea (for workspace obstacle S) 1. Find a point in S’s C-obstacle (robot placement colliding with S) 2. Select a random direction in C-space 3. Find a free point in that direction 4. Find boundary point between them using binary search (collision checks)

Note: we can use more sophisticated heuristics to try to cover C-obstacle

C-obst

Applications

Same Method: Diverse range of problems

•  Intelligent CAD •  digital testing, training, and servicing

• Simulating Group Behaviors •  homing, covering, shepherding

•  Modeling Molecular Motions •  protein folding, RNA folding, protein/ligand binding

PHANToM 1. User collects approximate path using

haptic device •  User insight identifies critical cfgs •  User feels when robot touches obstacles

and adjusts trajectory 2. Approximate path passed to planner and

it fixes it •  Planner is more efficient because search is

targeted to promising areas

Current Applications •  Intelligent CAD Applications •  Molecule Docking in drug design •  Animation w/ Deformable Models

Hybrid Human/Planner System: Generating and Repairing Approximate Paths [IEEE ICRA 01, 0; PUG’99]

C-obstacle

C-obstacle approximate

path

repaired path

0

500

1000

1500

2000

2500

3000

3500

time (sec)

OBPRMhaptic pushiterative push

Hybrid Human/Planner System Haptic Hints Results

Flange Problem 0.85 0.95 1

1.0 (original size)

.95 (95% of original size)

Applications

Same Method: Diverse range of problems

•  Intelligent CAD •  digital testing, training, and servicing

• Simulating Group Behaviors •  homing, covering, shepherding

•  Modeling Molecular Motions •  protein folding, RNA folding, protein/ligand binding

Group Behaviors J. Lien, B. Bayazit, S. Rodriguez, R. Sowell [PG’02,ALIFE’02,WAFR’02,ICRA04, ICRA05]

Solution: Roadmap-based flocking!

  Flocking systems are good at simulating behaviors of groups of objects (schools of fish, crowds…) –  flock formation is selfish, local,

decentralized, and efficient

  Flocking systems are not good at complex navigation or customizing behavior in different regions –  but roadmap-based planners are! ….

but generally just for one robot….

Roadmap-based Flocking

  Roadmap –  Map encoding global information (e.g., topology) –  data structure for storing and accessing

information –  supports implicit communication among group –  Customize agent behavior in different regions

  Agents –  have traditional flocking behavior, local sensing

ability –  have memory & reasoning –  dynamically (locally) select routes in roadmap

  edges selected based on edge weights   Edge weights updated as agents traverse them (ant

pheromone)

?

?

Covering -50 flock members

- Environment size: 80 m *100 m

- 16 obstacles

- Sensory range: 5 m

Covering Movie.

Covering Rule

Go to the less visited edge

Group Behaviors: Roadmap-Based Flocking

Emergency Response •  Evacuation Planning and Training

•  placement of personnel and barriers

- 3 available exits - no directors or barriers

- 1 available exit (2 blocked) - 2 barriers placed at blocked exits

Applications

Same Method: Diverse range of problems

•  Intelligent CAD •  digital testing, training, and servicing

• Simulating Group Behaviors •  homing, covering, shepherding

•  Modeling Molecular Motions •  protein folding, RNA folding, protein/ligand binding

Protein Folding   We are interested in the folding process

－  How the protein folds to its native, stable structure

  Importance of Studying Pathways –  insight into protein interactions & function

–  may lead to better structure prediction algorithms –  Diseases such as Alzheimer’s & Mad Cow related to

misfolded proteins

  Computational Techniques Critical –  Hard to study experimentally (happens too fast) –  Can study folding for thousands of already solved

structures –  Help guide/design future experiments

normal - misfold

prion protein

Folding Landscapes

  Each conformation has a potential energy –  Native state is global

minimum   Set of all conformations

forms landscape   Shape of landscape

reflects folding behavior

Configuration space

Potential

Native state

Different proteins different landscapes different folding behaviors

Using Motion Planning to Map Folding Landscapes [RECOMB 01,02, 04, 06, 07; ISMB 07; JMB 08; PSB 03]

A conformation

Configuration space

Potential

  Use Probabilistic Roadmap (PRM) method from motion planning to build roadmap

  Roadmap approximates the folding landscape –  Characterizes the main

features of landscape –  Can extract multiple

folding pathways from roadmap

–  Compute population kinetics for roadmap

Native state

Challenging Case Study [Thomas, Tang, Tapia, Amato - RECOMB 06 & 07, ISMB 07] Protein G

α, β3β4, β1β2, β1β4 [Kuszewski et al 1994, Orban et al. 1995]

Protein L

α, β1β2, β3β4, β1β4 L [Yi & Baker 1996, Yi et al 1997]; NuG1, NuG2 [Nauli, Kuhlman, Baker 2001]

NuG1 NuG2

•  Proteins G, L and NuG1 & NuG2 (mutated forms of G) •  All have similar structure (1 helix, 2 beta strands) •  G & L have only 15% sequence identity (G, NuG1, NuG2 similar sequence) •  In G beta 3-4 forms before beta 1-2, vice versa in L, NuG1, & NuG2

•  Can our approach detect the difference? Yes! •  When analyzing all (~10K) paths in our roadmaps

•  99% Protein G paths ,100% Protein L paths have “right” order •  98% and 97% paths in NuG1 and NuG2 have “right” order

Conclusions

Diverse problems can be addressed using appropriate adaptations of PRMs

•  key is defining appropriate models and their C-spaces •  validation check is very general - ranging from traditional collision detection to potential energy thresholds to ...

More info and Movies: http://parasol.tamu.edu/~amato amato@cs.tamu.edu

Joint work with -  my former students: O. Burchan Bayazit (WUSTL), Jyh-Ming Lien (George Mason), Marco Morales (ITAM, Mexico), Guang Song (Iowa State), Xinyu Tang (Google) - current students: Phil Coleman, Jory Denny, Roger Pearce, Sam Rodriguez, Robert Salazar, Lydia Tapia, Shawna Thomas, and the other members of our group -  other colleagues: Ken Dill (UCSF), David Giedroc (Indiana), Marty Scholtz