Path Following & Path Following & Motion ControlMotion Control

Stephen J. GuyStephen J. Guy

One Theme – Two paperOne Theme – Two paper

►Pedestrian path following Pedestrian path following Control Control usingusing physics physics

►Controlling Physical SimulationsControlling Physical Simulations Control Control ofof physics physics

Reactive Pedestrian Reactive Pedestrian DynamicsDynamics

Reactive Pedestrian Reactive Pedestrian DynamicsDynamics

► Premise: Premise: Building visualizations are more Building visualizations are more

compelling with moving peoplecompelling with moving people Need a simple to place moving virtual Need a simple to place moving virtual

people in modelspeople in models► Approach:Approach:

1.1. Let users specify pathLet users specify path2.2. Make agents follow it naturallyMake agents follow it naturally

Reactive Pedestrian Path Following from Reactive Pedestrian Path Following from Examples, Ronal Metoyer Oregon State, Examples, Ronal Metoyer Oregon State, Jessica Hodgins CMU (2004)Jessica Hodgins CMU (2004)

Agent ModelAgent Model

►Create “Intelligent” Agents to follow Create “Intelligent” Agents to follow pathpath

► Intelligence comes from force models Intelligence comes from force models in the environment and on other in the environment and on other agentsagents

►Example forces:Example forces:

User Defined PathsUser Defined Paths

►Exploits humans’ natural ability to Exploits humans’ natural ability to general realistic pathsgeneral realistic paths

►Convert paths into forces:Convert paths into forces:

f(s) – Unit vector along path (at nearest point)f(s) – Unit vector along path (at nearest point) f(df(dmax) – unit vector perpendicular to f(s)max) – unit vector perpendicular to f(s)

pp – the perpendicular distance to the path – the perpendicular distance to the path

dd – max is the largest allowed distance– max is the largest allowed distance

kkp p – the path gain– the path gain

Path Following Diag.Path Following Diag.

► Example application of forcesExample application of forces Note the direction of force changes with distance from pathNote the direction of force changes with distance from path

Hazard’s of Potential Based Hazard’s of Potential Based ModelsModels

►The authors state: The authors state: ““The intelligence model alone should produce correct 2D motion The intelligence model alone should produce correct 2D motion in terms of avoiding the defined obstacles and reaching goals, in terms of avoiding the defined obstacles and reaching goals, but will not necessarily produce natural motion for navigating but will not necessarily produce natural motion for navigating complex scenes.”complex scenes.”

►Reading between the lines:Reading between the lines: Our approach (like any potential based method) can Our approach (like any potential based method) can

mess up when two dynamic agents are involvedmess up when two dynamic agents are involved► Solution – Let the user fix itSolution – Let the user fix it

User warned of collisions and asked how to handle:User warned of collisions and asked how to handle:►YieldYield►Cut in frontCut in front►Go-around-right (or left)Go-around-right (or left)►No actionNo action

Implementing Avoidance Implementing Avoidance PrimitivesPrimitives

► Cut-In Front and Yield:Cut-In Front and Yield: Change the agent’s Change the agent’s

speed to avoid collisionspeed to avoid collision

►Go Around Right:Go Around Right: Change the agent’s Change the agent’s

direction to avoid direction to avoid collisioncollision

Automating Avoidance Automating Avoidance PrimitivePrimitive

►Somehow we’ve moved away from our Somehow we’ve moved away from our goal of an automatic system for path goal of an automatic system for path followingfollowing

►Key Idea - Learn from previous user Key Idea - Learn from previous user corrections (Machine Learning/AI)corrections (Machine Learning/AI)

►Record data needed to repeat user Record data needed to repeat user decisions:decisions: Layout of scene Layout of scene Position of agentsPosition of agents Speed of agentsSpeed of agents

TrainingTraining

► Build a naïve Bayes classifierBuild a naïve Bayes classifier► Inputs: Inputs:

Is the path around left blocked (Y or N)Is the path around left blocked (Y or N) Is the path around right blocked (Y or N)Is the path around right blocked (Y or N) Relative speed of the colliding pedestrians (5)Relative speed of the colliding pedestrians (5) Approach direction of the colliding pedestrians Approach direction of the colliding pedestrians

(8)(8) Colliding pedestrian’s distance to collision (5)Colliding pedestrian’s distance to collision (5) Pedestrian’s distance to collision (5)Pedestrian’s distance to collision (5) Desired travel direction (3)Desired travel direction (3)

► Outputs:Outputs: YieldYield Cut in frontCut in front Go-around-right (or left)Go-around-right (or left) No actionNo action

► aajj – j – jthth actionaction► hhii – i – ithth avoidance technique avoidance technique► H – set of all avoidance techniquesH – set of all avoidance techniques► nnhihi – number of times user selected h – number of times user selected hii

► |N| – total number of user selections|N| – total number of user selections

ResultsResults

►No videos (dead links No videos (dead links ))►Simulation snapshot -->Simulation snapshot -->►Bayes ClassificationBayes Classification

Sampling Plausible Sampling Plausible Solutions to Multi-body Solutions to Multi-body

Constraint ProblemsConstraint Problems

Solution Sampling for Constraint Solution Sampling for Constraint SatisfactionSatisfaction

► PremisePremise Extreme sensitivity makes multi-body simulations Extreme sensitivity makes multi-body simulations

difficult to controldifficult to control Users general want a goal, more than the care how Users general want a goal, more than the care how

the goal is achievedthe goal is achieved► ApproachApproach

Allow users to specify constraints (goals)Allow users to specify constraints (goals) Explore simulation spaceExplore simulation space Return animations which likely satisfied constrainsReturn animations which likely satisfied constrains

Chenney, S., and Forsyth, D. A. 2000. Sampling plausible solutions to multi-body constraint problems. In Proceedings of ACM SIGGRAPH 2000, 219–228.

Simulation RandomnessSimulation Randomness

►Simulation Simulation needsneeds randomness for this randomness for this technique to worktechnique to work Needs variations in simulations to exploreNeeds variations in simulations to explore

►Authors argue randomness is good Authors argue randomness is good anywayanyway Models small variations in otherwise flat Models small variations in otherwise flat

surfacessurfaces Closer to real worldCloser to real world Closer to user expectation Closer to user expectation

Incorporating UncertaintyIncorporating Uncertainty

►Define function pDefine function pww(A) (prob. of (A) (prob. of animation A) which is low for unlikely animation A) which is low for unlikely As and high for likely AsAs and high for likely As

►Consider a bouncing ball (2D)Consider a bouncing ball (2D)►A is collection of normals A is collection of normals ii

►Vertical normals likely: Vertical normals likely:

Constraints SatisfactionConstraints Satisfaction

►We want pWe want pww(A | C) ‘s distribution in A(A | C) ‘s distribution in A Return animations with high likely in pReturn animations with high likely in pww

►Example ConstraintsExample Constraints Ball must go from here to thereBall must go from here to there Ball much have final velocity of fooBall much have final velocity of foo Ball must bounce through hoopBall must bounce through hoop

►Finding pFinding pww(A|C) is not (generally) (A|C) is not (generally) possiblepossible

Constrain ApproximationConstrain Approximation► Can’t use pCan’t use pww(A|C)(A|C)

► Instead use p(A) = pInstead use p(A) = pww(A)*p(A)*pcc(A)(A) ppcc(A) depends on how well A satisfies constraints(A) depends on how well A satisfies constraints

► For ball example For ball example (d is target distance form center)(d is target distance form center)

► AndAnd

► Sample according to p(a)Sample according to p(a) Mainly likely eventsMainly likely events Occasionally unlikely eventsOccasionally unlikely events

Defining PlausibilityDefining Plausibility

►Moves past “it it looks right to me its Moves past “it it looks right to me its right”right”

►Plausibility is statistically defined be Plausibility is statistically defined be P(A)P(A)

►Possible to validate important parts of Possible to validate important parts of P(A) against a real world modelP(A) against a real world model

►But there are other approachesBut there are other approaches (See next paper)(See next paper)

Markov Chain Monte Carlo Markov Chain Monte Carlo (MCMC)(MCMC)

►Markov Chain ModelMarkov Chain Model Given a sequence of events, next state is Given a sequence of events, next state is

dependant dependant only only on current stateon current state►Monte Carlo AlgorithmsMonte Carlo Algorithms

Using random numbers to help (probably) Using random numbers to help (probably) solve problemssolve problems

►MCMCMCMC Generate sequence of animations: AGenerate sequence of animations: A11, A, A22, ,

AA33, …, … An dependant only on An-1An dependant only on An-1 A is distributed following p(A)A is distributed following p(A)

AlgorithmAlgorithm

1.1. initialize(Ainitialize(A00))

2.2. simulate(Asimulate(A00))

3.3. RepeatRepeat

4.4. propose(Apropose(Acc, A, Aii))

5.5. simulate(Asimulate(Acc))

6.6. u <- random(0,1)u <- random(0,1)

7.7. ifif u < min(1, ) u < min(1, )

8.8. AAi+1i+1 <- A <- Acc

9.9. elseelse

10.10. AAi+1i+1 <- A <- Aii

q(X | Y) is probability of transitioning from X to Y

Example – 2D BallExample – 2D Ball

► Generating AGenerating Ai+1i+1:: 50% of adding between 50% of adding between

-5 and 5 degrees to -5 and 5 degrees to each normal of the each normal of the previous animationprevious animation

► q(X|Y):q(X|Y):

► Choosing Choosing dd is tricky is tricky Too small – no variationToo small – no variation Too large – constraints Too large – constraints

aren’t well satisfiedaren’t well satisfied

Example - BowlingExample - Bowling

► Rank resulting pin-condition with Gibbs Rank resulting pin-condition with Gibbs distributiondistribution ppcc(A) = (A) = k+mk+m

► > 1> 1►k number of correct pinsk number of correct pins►m number of undisturbed pinsm number of undisturbed pins► has the same constraints as has the same constraints as dd did before did before

Author’s claim less sensitivityAuthor’s claim less sensitivity

► Proposing new AProposing new Aii

Sample new values for all variablesSample new values for all variables Change the radius, density, or initial conditions of the Change the radius, density, or initial conditions of the

ballball Change the initial position of some pinsChange the initial position of some pins

Bowling – Videos (1/2)Bowling – Videos (1/2)

► 3 Different strike 3 Different strike animationsanimations Was easy to get strikes Was easy to get strikes

given simulationsgiven simulations

Bowling – Videos (2/2)Bowling – Videos (2/2)

Bowling 7-10 splitBowling 7-10 split

Example – Spelling BallsExample – Spelling Balls► Similar to bowlingSimilar to bowling

Gibbs distributionGibbs distribution Similar Parameters Similar Parameters

variedvaried

► Spelling “Hi”Spelling “Hi” ““A few hours” on an A few hours” on an

Pentium 200 MHzPentium 200 MHz

Example - DiceExample - Dice

► Approach from simple ball example won’t Approach from simple ball example won’t workwork Need a smooth change in normalsNeed a smooth change in normals

► Create “surface” using b-splinesCreate “surface” using b-splines► Vary control points of b-splineVary control points of b-spline►~ 1 hour per die~ 1 hour per die

Future WorkFuture Work

► Issues Not real time Not interactive (is this an issue?) Very slow Unclear how to best build p(A|C) (std. dev issues) Unclear how to best build the proposal function

► “Multi-body constraint problems are good candidates for a Design Galleries interface, in which a user browses through sample solutions to locate the one they prefer. Our work addresses the sampling aspect …, but we do not consider other aspects of the interface.”

Many-Worlds Browsing for Many-Worlds Browsing for Control of Dynamics Control of Dynamics

SimulationsSimulations

Many-Worlds Browsing for Many-Worlds Browsing for ControlControl

► PremisePremise Passive physical simulations are great…Passive physical simulations are great… But artists often need more controlBut artists often need more control Existing techniques are basicallyExisting techniques are basically

1.1. Convert user’s goals to a functionConvert user’s goals to a function

2.2. Minimize this function numericallyMinimize this function numerically

This is slow on non-linear simulationsThis is slow on non-linear simulations

► ApproachApproach Leverage human flexibility Leverage human flexibility Provide user options at interesting Provide user options at interesting

pointspointsChristopher D. Twigg and Doug L. James. Many-worlds Christopher D. Twigg and Doug L. James. Many-worlds browsing for control of multibody dynamics. browsing for control of multibody dynamics. ACM ACM Transactions on Graphics (SIGGRAPH 2007)Transactions on Graphics (SIGGRAPH 2007), 26(3), August , 26(3), August 2007.2007.

Keeping the User in the LoopKeeping the User in the Loop

►Previous work used offline Markov Previous work used offline Markov Chain Monte Carlo to reach goals Chain Monte Carlo to reach goals (Chenney and Forsyth) (Chenney and Forsyth)

►Authors argue it’s good to keep the Authors argue it’s good to keep the user in the loopuser in the loop Gives feedback on the simulationGives feedback on the simulation Even w/ ideal optimizations, user would Even w/ ideal optimizations, user would

need to give feedback on simulation setupneed to give feedback on simulation setup

Basic TechniqueBasic Technique

1.1. Specify initial simulation stateSpecify initial simulation state

2.2. Create various physical simulations, until a Create various physical simulations, until a collisioncollision

3.3. Report various results/options to userReport various results/options to user

4.4. Allow user to choose pathAllow user to choose path

5.5. Allow user to insert new objectsAllow user to insert new objects

6.6. RepeatRepeat

► It’s important to return a variety of results, It’s important to return a variety of results, but they all need to be but they all need to be plausibleplausible

PlausibilityPlausibility

► Studies have been performed on how much Studies have been performed on how much users notice wrong results [O’Sullivan]users notice wrong results [O’Sullivan]

► Variations users don’t mind/notice are Variations users don’t mind/notice are considered plausibleconsidered plausible

► Users were unable to detect distortions of up Users were unable to detect distortions of up to:to: 40% in magnitude of linear velocity40% in magnitude of linear velocity 20 degrees in direction of linear velocity20 degrees in direction of linear velocity 20% in angular velocity20% in angular velocity

► Assumes that chains of Assumes that chains of plausible actions plausible actions are also plausibleare also plausible

PracticalitiesPracticalities

►ParallelizationParallelization Running multiple simulations is slowRunning multiple simulations is slow Each simulation is independentEach simulation is independent Split over a cluster!Split over a cluster!

►Data transferData transfer Simulation from each comp. needs to be sent Simulation from each comp. needs to be sent

to userto user Need efficient representation (or lots of Need efficient representation (or lots of

bandwidth)bandwidth)►Represent the position as a piecewise quadratic Represent the position as a piecewise quadratic

splinespline►Represent the rotation as a piecewise linear skew-Represent the rotation as a piecewise linear skew-

symmetric matrixsymmetric matrix

User InteractionUser Interaction► Spatial queriesSpatial queries

Allow the user to section areas where the chosen Allow the user to section areas where the chosen objects should or should not go throughobjects should or should not go through

► Ranking metrics – allows the user to specity Ranking metrics – allows the user to specity additional metrics to sort valid queries byadditional metrics to sort valid queries by Angular velocity, running time, collisions, Angular velocity, running time, collisions,

orientation, desired constraintsorientation, desired constraints

► Refinement – allow user updates in realtimeRefinement – allow user updates in realtime

ResultsResults►UI was vary fast, able to show user over UI was vary fast, able to show user over

1,500 simulation results at once screen1,500 simulation results at once screen

►Creating interesting simulations was Creating interesting simulations was quickquick Refrigerator Toy – 5½ minutesRefrigerator Toy – 5½ minutes Spelling SIGGRAPH – 1 hour (w/ backtracking)Spelling SIGGRAPH – 1 hour (w/ backtracking) Spiral Staircase – 25 minutesSpiral Staircase – 25 minutes Trebuchet – Simulation took 14 hours so no Trebuchet – Simulation took 14 hours so no

realtime refinement was possiblerealtime refinement was possible

ResultResult

►SIGGRAPH VideoSIGGRAPH Video

►Live DemoLive Demo

ComparisonComparison

► The three control techniques were radically The three control techniques were radically differentdifferent

► The first – worked to follow users path (with The first – worked to follow users path (with a physically inspired model)a physically inspired model)

► The second – gave user several options The second – gave user several options which fit their constraintswhich fit their constraints

► The third – allowed the user to naturally The third – allowed the user to naturally explore physically based options explore physically based options

► Some similaritiesSome similarities Leverage user input when it makes senseLeverage user input when it makes sense Physics! (making life better)Physics! (making life better)

QuestionsQuestions

??

ReferencesReferences

Ronald Metoyer, and Jessica Hodgins, Reactive Pedestrian Path Ronald Metoyer, and Jessica Hodgins, Reactive Pedestrian Path Following from Examples, (2004).Following from Examples, (2004).

Christopher D. Twigg and Doug L. James. Many-worlds browsing Christopher D. Twigg and Doug L. James. Many-worlds browsing for control of multibody dynamics. for control of multibody dynamics. ACM Transactions on ACM Transactions on Graphics (SIGGRAPH 2007)Graphics (SIGGRAPH 2007), 26(3), August 2007., 26(3), August 2007.

Chenney, S., and Forsyth, D. A. 2000. Sampling plausible solutions to multi-body constraint problems. In Proceedings of ACM SIGGRAPH 2000, 219–228.

O’Sullivan, C., Dingliana, J., Giang, T., And Kaiser, M. K. 2003. Evaluating the visual fidelity of physically based animations. ACM Transactions on Graphics 22, 3 (July), 527–536.