![Page 1: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/1.jpg)
Spacetime Constraints Revisited
J. Thomas NgoGraduate Biophysics
ProgramHarvard University
Joe marksCambridge Research Lab
Digital Equipment Corporation
![Page 2: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/2.jpg)
Spacetime Constraints
• Indirectly controlling physically realistic motion of articulated figures
• Animator Defines:– Physical structure– Actuators– Criteria for evaluating motion
• Computer computes optimal trajectory according to specified criteria
![Page 3: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/3.jpg)
Globally Optimal Solutions
• Current SC techniques are local in nature
• Finding Globally Optimal Solutions to SC Problems is tough– Multimodality
• Exponential number of possible trajectories
– Many may be locally optimal or near optimal
– Search-space discontinuities• Small change in actuators causes large
change in trajectory
![Page 4: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/4.jpg)
A Way to Find Globally Optimal Solutions:• Global search algorithm
– Generates multiple near-optimal trajectories• Encode trajectories as sets
of stimulus-response behavior rules
• Genetic algorithm for choosing behavior parameters
![Page 5: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/5.jpg)
The Algorithm
• Dynamics module creates physically correct environment.
• Behavior module generates creature behaviors
• Search module finds values for stimulus-response parameters.
![Page 6: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/6.jpg)
Dynamics Module
• Uses forward dynamics– Find motions from forces
• Articulated figure treated as autonomously deforming object with no DOF– Deformations produced
kinematically by stimulus-response control algorithm.
– Save CPU cycles.
![Page 7: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/7.jpg)
Dynamics Module
• Friction is static when one joint touches floor
• Slippage proportional to contact force when two joints touchFriction
?
![Page 8: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/8.jpg)
Behavior Module
• Uses parameterized algorithm based on stimulus and response
• Stimulus-Response Control Algorithm– Reflexes triggered by
conditions sensed in environment• Conditions – stimulus functions• Reflexes - responses
![Page 9: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/9.jpg)
Behavior Module - Responses
• Response – direction for changing creature shape smoothly.– Time Constant– Set of Target values for
creature’s internal angles
![Page 10: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/10.jpg)
Behavior Module - Responses
• Throw values into critically damped equation of motion for each angle
• time constant• target value of angle• actual value of angle
• Creature approaches target shape smoothly.
0)(2 02 iiii
0i
i
![Page 11: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/11.jpg)
Behavior Module - Stimulus Functions
• Sense variable– Real-valued function of environment
• Proprioceptive senses – Um, joint angles
• Tactile senses– Force exerted by endpoint on floor
• Kinestheitic sense– Vertical velocity of center of mass
• Position sense– Vertical position of center of mass
![Page 12: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/12.jpg)
Behavior Module - Stimulus Functions• Stimulus functions
– Scalar function defined over sense space
V
jjjj vvW
1
20 )(max1
0jv
j
Sense Variables
Determined by search module
Determined by search module
Weight
V
j j
jW1
minlog
},...,,{ 21 Vvvv
![Page 13: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/13.jpg)
Behavior Module - Stimulus Functions
• Constants normalized so sense variables fall between 0 and 1
• Sensitive Region– Locus of points in sense space
where stimulus-function is positive• Forms a hyper-rectangle
– Dimensions
– Centered at
V2,...,2,2
21
002
01 ,...,, Vvvv
![Page 14: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/14.jpg)
Behavior Module - Stimulus Response Functions• Set of SR parameters consists of array of SR
pairs• To generate behavior:
Initialize creature state from SC problem descriptionActivate response 0for t = 1 to T
Determine deformation for time t from active responseSimulate resulting dynamics for time tMeasure sense variables from the environmentIdentify highest-valued stimulus functionActivate corresponding response if stimulus positive
end for
• Active response changed only if highest-valued stimulus function is positive– Causes response to be active for several time step
• Coherent motion
![Page 15: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/15.jpg)
Search Module – Genetic Algorithm• Parallel genetic algorithm
– Written in C* on Thinking Machines CM-2 with 4096 processors
• Each processor evaluates one genome per generation
![Page 16: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/16.jpg)
Search Module
• Q: How to find values for stimulus and response parameter?
• A: Trial and Error. (Genetic Algorithm)
![Page 17: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/17.jpg)
Search Module – Genetic Algorithm• Pseudocode:do parallel
Randomize genomeend dofor generation = 1 to number_of_generations
do parallelEvaluate genomeSelect mate from another processorCross genome with mateMutate genome
end do end for
![Page 18: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/18.jpg)
Search Module – Genetic Algorithm
• Parameters of one SR pair in genome
![Page 19: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/19.jpg)
Search Module – Genetic Algorithm• Randomization
– All parameters initially chosen at random•Uses probability distribution
– Hill climbing to enrich initial gene pool•Evaluate initial gene pool•Mutate and re-evaluate solution 4 times•On each processor choose best out of five
– Makes population skewed in favor of multi-step behaviors
![Page 20: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/20.jpg)
Search Module – Genetic Algorithm
• Mate Selection– Processors laid on in imaginary 64x64 grid– Each processor does 10 step random walk– If best solution is itself, no mating– Else mates with best encountered genome
• Good genes diffuse through population• Spreading is not instantaneous so inferior solutions get some processing time• Population converged when one population dominates
![Page 21: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/21.jpg)
Search Module – Genetic Algorithm• Crossover
– Typical linear layout for crossover not meaningful
– Crossover must be tailored to problem to get good performance
![Page 22: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/22.jpg)
Search Module – Genetic Algorithm• Mutation
– Tailored specifically for the SR representation.• One SR Pair creeps• One SR Pair randomized form scratch
– But at least one corner of sensitive region of new stimulus function must coincide with original sense-space trajectory.
» Else new stimulus functions dominate trajectory or don’t do anything
![Page 23: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/23.jpg)
Results – Five-Rod Fred
• Five-Rod Fred– 5 consecutively linked rods
• Middle rods have same mass• Terminal rods five times heavier
• Evaluation Function – Move COM as far right as possible
• Expected inch-worm behavior
• But…
![Page 24: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/24.jpg)
Results – Five-Rod Fred
After 64 generations…
Curling leap
![Page 25: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/25.jpg)
Results – Five-Rod Fred
After 100 generations…
Curling leap w/ a roll
![Page 26: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/26.jpg)
Results – Five-Rod Fred
• Final behavior generated by 5 of 10 SR pairs– Two pairs produce initial
curling motion– Two pairs produce leaping
motion– One pair produces final
curled shape
![Page 27: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/27.jpg)
Results – Mr. Star-Man
• Mr. Star-Man– Five rods in star shape– All rods have equal length,
mass• Evaluation Function –
Move COM as far right as possible
![Page 28: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/28.jpg)
Results – Mr. Star-Man
After 20 generations…
Sideways cantering
![Page 29: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/29.jpg)
Results – Mr. Star-Man
After 37 generations…
Ghetto tripping cartwheel
![Page 30: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/30.jpg)
Results – Mr. Star-Man
After 94 generations…
Sideways shuffling
![Page 31: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/31.jpg)
Results – Beryl Biped
• Beryl Biped– Headless 2D humanoid
• Rigid torso• Jointed legs• Point feet• Rod masses of human proportion
• Evaluation Function – Move point between feet as far right as possible– If used COM, just fell down
![Page 32: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/32.jpg)
Results – Beryl Biped
After 100 generations…
Skipping, back leg never gets In front of forward leg
![Page 33: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/33.jpg)
Results – Beryl Biped
Another trial…
Skipping walk
![Page 34: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/34.jpg)
Results – Beryl Biped
Another trial…
Weird walk
![Page 35: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/35.jpg)
Results
• GA will fail to find good optima of evaluation function is no good
• Small changes can restore expected behavior
• How to change evaluation function to reward grace?
???
![Page 36: Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation](https://reader036.vdocuments.mx/reader036/viewer/2022062401/5a4d1b0b7f8b9ab05998b0c1/html5/thumbnails/36.jpg)