Download - Computational Motor Control: Optimal Control for Deterministic Systems (JAIST summer course)
Computational Motor Control Summer School02: Optimal control for deterministic systems
Hirokazu Tanaka
School of Information Science
Japan Institute of Science and Technology
Optimal control for deterministic (noiseless) systems.
In this lecture, we will learn:
• Invariant laws of body movements
• Calculus of variation
• Lagrange multiplier method
• Karush-Kuhn-Tucker condition
• Pontryagin’s minimum principle
• Boundary problem
• Minimum-Jerk model
• Minimum-torque-change model
• Kinematic vs dynamic planning in reaching movements
Straight paths and bell-shaped velocity in reaching.
Morasso (1981) Exp Brain Res
12
3 4 5
6
joint angles
angular velocity
angular accel.
hand speed
T1->T4 T3->T5 T2->T5 T1->T5
Power laws for curved movements.
Lacquaniti et al. (1983) Acta Psychologica
2
3 1
3v
: angular velocity
: h
: curvature
and speedv
or
curvature
angu
lar
velo
city
Fitts’ law for movement duration in rapid pointing movements.
Fitts (1954) J Exp Psychol
2
2log
f
Dt a b
W
“Main sequence” for saccadic movements
Bahill et al. (1975) Math Biosci
Optimality principle: how a unique trajectory is chosen.
http://en.wikipedia.org/wiki/Snell%27s_law
Snell’s law
Fermat’s principle of least time
1 1 1
2 2 2
sin 1/
sin 1/
v n
v n
Q Q
0 P P
1T dsT s dt n s ds
v s c
Q
P
10T s n s ds
c
Optimality principle: how a unique trajectory is chosen.
Newton’s equation of motion
Hamilton’s principle of least action
Vm
q
2
1
,t
tS dtL q q q
21,
2L V q q q q
0S q
http://en.wikipedia.org/wiki/Principle_of_least_action
Mathematics: Calculus of variation.
0
, ,ft
xS x x L x dt
0
0 0
0
0
, ,
, , ,
f
f f
f
f
t t
t
t t
t
t
L x dt L x dt
L
S x x S x x x x S x x
x
Ldt
x x
L d L Ldt
x dt
x x x
x
x
x
x xx
0d L L
dt x x
0
0
ft t t
L L
xx
xx
S: Action, L: Lagrangian
Euler-Lagrange equation
x t x t x t variation
Mathematics: Calculus of variation.
0
, ,, ,, ,ft
S x dtLx xx xxx x
0 0
0
2 3
2 3
2
0
2
, , , , ,, ,, ,f f
f
f
t t
t
t L d L d L d L
x dt x dt x dt
L d L d Lx
x
S x x x x x xx dtL x dtL x
dt
dt x dt x
x x x x x x
x
x
x
00
f ft t
L d L Lx x
x dt x x
2 3
2 40 0 f
L d L d L d Lt t
x dt x dt x dt x
2
2
000
0
f f ft t t
L d L d L L d L Lx x x
x dt x dt x x dt x x
Lagrangian with higher derivatives
Euler-Poisson equation
Mathematics: Calculus of variation.
2 3
2 30 0 f
L d L d L d Lt t
x dt x dt x dt x
2
2
000
0
f f ft t t
L d L d L L d L Lx x x
x dt x dt x x dt x x
2 3
2 30 0 f
L d L d L d Lt t
x dt x dt x dt x
0 0 00f f ft t t
x x x
Euler-Poisson equation with general boundary conditions
Euler-Poisson equation with fixed boundary conditions
Smoothness criterion: Minimum-jerk model.
2 2
3
30
3
3MJ0
, , , , , , ,ftft d x d y
x x x x y y y y dt dtdt dt
C L
Flash & Hogan (1985) J Neurosci
2 3 3
2 3 6
6
30 2 2
L d L d L d L d d xx
x dt x dt x dt x dt dt
6
6 6
60
d x d y
dt dt
Intuition: The observed movement trajectories are smooth, so “smoothness” of trajectory may be selected in the brain.
: position, : velocity, : acceleration, : jerkx x x x
Smoothness criterion: Minimum-jerk model.
Flash & Hogan (1985) J Neurosci
6
6 6
60
d x d y
dt dt
00
f f
x x
x t x
0 0 0
0f f
x x
x t x t
Boundary conditions for a point-to-point movement:
Euler-Lagrange equation:
45
0 0
3
6 15 10f
f f f
t t tx xt x x
t t t
Solution of minimum-jerk trajectory (5th order polynomial)
Smooth trajectory and bell-shaped velocity explained by the model.
Flash & Hogan (1985) J Neurosci
speed y accel x accel speed y accel x accel
Via-point movements also explained by the model.
Flash & Hogan (1985) J Neurosci
3 4 5
0 1 2
2
3 4 5 10x t a a t a t a t a t a t t t
3 4 5
0 1 2 3 4
2
5 1 fx t a a t a t a t a t a t t tt
00 , 0 0 0x x x x
, 0f f ffx t x x t x t
1 1 1 1 1 1
1 1 1 1 1 1
, , ,
, , .
x t x x t x x t x t
x t x t x t x t x t x t
00x x
1 1x t x
f fx t x
x t
x t
Twelve unknown coefficients can be determined by twelve boundary conditions.
ia
Power law predicted by the minimum-jerk model.
Viviani & Flash (1995) J Exp Psychol
Smoothness criterion: path-constrained Minimum-jerk model.
Huh & Sejnowski (2015) PNAS
Cartesian coordinates Frenet-Serret coordinates
ˆv v t
4 26 ˆ ˆ32 1v z z h z t z n
v v
h
0 0
log , logv
z hv
Smoothness criterion: path-constrained Minimum-jerk model.
Huh & Sejnowski (2015) PNAS
4 26 ˆ ˆ32 1v z z h z t z n
v v
h
2
20
d
dzdz z
d
4 6 2 3 4
3
2 2 2
2 2
5 2 30 10 25 82 40
2 14 90 12
20 55 75 15
82 8 22 20 0 129
v z z h h h z z
z h h h h h
z h h z
z z h h zh z z h
Minimum-jerk Lagrangian in Frenet-Serret coordinates:
Derive Euler-Lagrange equation:
Or explicitly:
Derivation of two-thirds power law.
Huh & Sejnowski (2015) PNAS
4 6 2 3 4
3
2 2 2
2 2
5 2 30 10 25 82 40
2 14 90 12
20 55 75 15
82 8 22 20 0 129
v z z h h h z z
z h h h h h
z h h z
z z h h zh z z h
0
ae
0
logh a
0
bv ev
0
logv
v bv
Euler-Lagrange equation:
Spiral path:
or
Assume a solution in an exponential form:
or
4 6 2 3 4 3 2 2
0
4 6
4 6
0 5 25 82 40 90 12 55
(constant)
75a b
a b
v e a ab b b a a b a
e
Substituting the exponential forms into the Euler-Lagrange equations:
2
3b a
Model predicts the power law in spiral movements.
Huh & Sejnowski (2015) PNAS
0
ae
2
30ev v
therefore, 2/3v
That’s what was found in experiment!
Scaling law with figure-dependent exponents: model prediction and experimental confirmation.
Huh & Sejnowski (2015) PNAS
sinh
Optimization with equality constraints: Lagrange multiplier method.
Minimize a function f(x) under a constraint g(x)=0.
0J f g
x x x
0J
g
x
Image source: Wikipedia
,J f g x x x
λ: Lagrange multiplier
Necessary condition:
Karush-Kuhn-Tucker (KKT) condition for constrained optimization.
Minimize a function f(x) under an inequality condition .
,J f g x x x
0
0
0
0
J f g
g
g
x
x
x x
x
Image source: Wikipedia
0g x
λ: Lagrange multiplier
KKT condition:
Optimization under dynamic constraint: Pontryagin’s minimum principle.
,x f x u
0
,ft
fJ g dt u x x u
0 0
0
0
T
T T
T
, , , ( , )
, ,
, ( , )
f f
f
f
t t
t
t
f
f
f
J g dt dt
g dt
dt
x u p x x u p x f x u
x x u p f x u p
x xu p
x
x p
Minimize:
under constraint of EOMs:
T, , , ,g x u p x p ufu xHamiltonian
Optimization under dynamic constraint: Pontryagin’s minimum principle.
0
0
T T
TT T
, , , , , ,
, , , ,f
f
f
f
t
t
f f
T
t
t
x
J J J
dt
dt
x u p
x u p x u p p x
x p ux x p
x u p x u p x u p
x x x
x u p p x
u
p x
p p x
T
,
0
g
p
f
x x x
x f x u
p
u
p
f
f
t t
t
xp
EOMs
Terminal condition
Smoothness criterion: Minimum-torque change model.
T
1 2 1 2 1 2 x
22
2
2 2 2 2
11
1 2 1 11
2 1
11
22
,
,
, ,,
, ,
f
f
u
u
x f x u
0 0
T T1 1
2 2
f ft t
J dt dt τu u uτ
T T1, , ,
2 x u p u u p f x u
State vector
EOMs
Torque-change cost
Uno, Kawato & Suzuki (1989) Biol Cybern
Smoothness criterion: Minimum-torque change model.
Uno, Kawato & Suzuki (1989) Biol Cybern
T
T
,
0
x f xp
f
x x
fu
u u
u
p p
p
00 x x
00 p p
Initial condition for x:
Initial condition for p:
p0 must be chosen so that .fft x x
Smoothness criterion: Minimum-torque change model.
Uno, Kawato & Suzuki (1989) Biol Cybern
Experiment Model
Smoothness criterion: Minimum-torque change model.
Uno, Kawato & Suzuki (1989) Biol Cybern
Experiment Model
Smoothness criterion: Minimum-torque change model.
Uno, Kawato & Suzuki (1989) Biol Cybern
Experiment Model
Is movement planning in extrinsic or intrinsic space?
Wolpert et al. (1993) Exp Brain Res
Experiment
Visual perturbation experiment: MJ prediction: adapted path under visual perturbationMTJ prediction: non-adapted path under visual perturbation.
Summary
• Human movements exhibit a variety of invariant features (straight paths, power law, Fitts’ law, main sequence, …).
• Those invariant features are explained in terms of optimality principles.
• There are mathematical methods for solving optimization problems (calculus of variation, Lagrange multiplier methods, Pontryagin’s minimum principle, etc.).
• Smoothness in trajectory or joint torques is one of the most successful criterion for reaching movements.
References
• Morasso, P. (1981). Spatial control of arm movements. Experimental Brain Research, 42(2), 223-227.
• Lacquaniti, F., Terzuolo, C., & Viviani, P. (1983). The law relating the kinematic and figural aspects of drawing movements. Acta Psychologica, 54(1), 115-130.
• Fitts, P. M. (1954). The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47(6), 381.
• Bahill, A. T., Clark, M. R., & Stark, L. (1975). The main sequence, a tool for studying human eye movements. Mathematical Biosciences, 24(3), 191-204.
• Flash, T., & Hogan, N. (1985). The coordination of arm movements: an experimentally confirmed mathematical model. The journal of Neuroscience, 5(7), 1688-1703.
• Viviani, P., & Flash, T. (1995). Minimum-jerk, two-thirds power law, and isochrony: converging approaches to movement planning. Journal of Experimental Psychology: Human Perception and Performance, 21(1), 32.
• Huh, D., & Sejnowski, T. J. (2015). Spectrum of power laws for curved hand movements. Proceedings of the National Academy of Sciences, 112(29), E3950-E3958.
• Uno, Y., Kawato, M., & Suzuki, R. (1989). Formation and control of optimal trajectory in human multijoint arm movement. Biological Cybernetics, 61(2), 89-101.
• Wolpert, D. M., Ghahramani, Z., & Jordan, M. I. (1995). Are arm trajectories planned in kinematic or dynamic coordinates? An adaptation study. Experimental Brain Research, 103(3), 460-470.
• Flanagan, J. R., & Rao, A. K. (1995). Trajectory adaptation to a nonlinear visuomotor transformation: evidence of motion planning in visually perceived space. Journal of neurophysiology, 74(5), 2174-2178.
Exercise
• Point-to-point minimum-jerk solution: For given initial and final positions, draw a minimum-jerk trajectory (path and velocity).
• Via-point minimum-jerk solution: Find a via-point trajectory by determining the twelve coefficients with given boundary conditions.
• Write a MATLAB code to solve the two-boundary problem of the minimum-torque change model.