Sam Burdencurrently: Postdoc in EECS at UC BerkeleySep 2015: Asst Prof in EE at UW Seattle

Dynamic Inverse Models in Cyber-Human Systems

S. Shankar SastryDean of Engineering at UC Berkeley

Embedded Humans: Provably Correct Decision Making for Networks of Humans and Unmanned Systems (N00014-13-1-0341)

Increasingly mediated by automation– Augmented by hardware and software– Machines adapt to collaborate and assist

Human interaction with the physical world

Cyber-Human Systems

Embedding humans amid automationCan lead to performance degradation

– pilot-induced oscillations in rotory/fixed-wing aircraftMcRuer, Krendal 1974; Hess J. Guid. Cont. Dyn. 1997Pavel et al. Prog. Aero. Sci. 2013

– overreliance on adaptive cruise control in carsRudin-Brown, Parker Trans. Rch. F: Traffic Psych. and Behav. 2004

Requires predictive models for human behavior

Behavioral repertoire of humansToo rich to model from first principles

– Spans computational, algorithmic, & physical “levels of analysis”Marr, Poggio MIT AI MEMO 1976

– Influenced by neurophysiological state (cognitive load, hunger)LaPointe, Stierwalt, Maitland Int. J. Speech-Lang. Pathology 2010; Danziger, Levav, Avnaim-Pesso PNAS 2011

Can reduce dramatically during particular tasks– Bernstein posed the “problem of motor redundancy”

Bernstein Pergamon Press 1967.

– Perhaps instead we “exploit the bliss of motor abundance” e.g. using synergies, uncontrolled manifolds, optimality

Latash Exp. Brain Rch. 2012; Ting, Macpherson J. Neurophys. 2005; Scholz, Schoner, Exp. Brain Rch. 1999Todorov, Jordan Nature Neurosci. 2002; Diedrichsen, Shadmehr, Ivry Trends Cog. Sci. 2010

– For instance, locomotion naturally reduces dimensionalityBurden, Revzen, Sastry IEEE TAC 2015

Predictable behavior from internal modelsPopular paradigm posits pairs of internal models

– forward model predicts sensory effect of motor actionSutton, Barto Psych. Rev. 1981; Jordan, Rumehlart Cog. Sci. 1992; Wolpert, et al. Science 1995

– inverse model computes motor command expected to yield desired behaviorKawato Curr. Opin. Neurobio. 1999; Thoroughman, Shadmehr Nature 2000; Conditt, Mussa-Ivaldi PNAS 1999

– Theoretical and empirical evidence for paired forward + inverse modelsBhushan, Shadmehr Bio. Cybern. 1999; Sanner, Kosha Bio. Cybern. 1999Hanuschkin, Ganguli, Hahnloser Front. Neural Circ. 2013; Giret, Kornfeld, Ganguli, Hahnloser PNAS 2014

Parallels in control theory, robotics, AI– Internal models, adaptive control, learning

Francis, Wonham Automatica 1976; Sastry, Bodson Prentice Hall 1989; Sutton, Barto, Williams IEEE CSM 1992Crawford, Sastry UCB EECS 1996; Atkeson, Schaal ICML 1997; Papavassiliou, Russell IJCAI 1999

Instantiating internal modelsForward model ( M : U Y ): static vs. dynamic

– static map (linear or nonlinear)Hanuschkin, Ganguli, Hahnloser Front. Neural Circ. 2013Giret, Kornfeld, Ganguli, Hahnloser PNAS 2014

– dynamic map depends on intermediate state

Thoroughman, Shadmehr Science 2000Wolpert, Diedrichsen, Flanagan Nature Neurosci. 2011

Inverse model ( M-1 : Y U ): hard to define– static map may fail to be one-to-one or onto– dynamic map may be acausal or need state estimate

Today’s talk: dynamic inverse models from the perspective of mathematical control theory

1. derivation of dynamic inverse model

2. properties and implications for cyber-human systems

Dynamic inverse models in cyber-human sys

Single input/single output forward modelConsider forward model in control-affine form:

– x in Rn, u in R, y in R– f, g in Cr(Rn, Rn), h in Cr(Rn, R)

Suppose model has strict relative degree in N:

– Expressed in terms of Lie derivatives Lf h(x), Lg h(x):

– intuitively, input affects -th derivative of output– e.g. =2 for Lagrangian mechanical systems

applicable to interaction with physical world

Transformation of forward modelForward model:Suppose model has strict relative degree in N:

– e.g. =2 for Lagrangian mechanical systemsThen model is linear in new coordinates:

– There exists such that in coordinates forward model has the form

– Choosing yields

simpler forward model

Dynamic inverse modelForward model:Given desired output yd, we seek desired input ud

– Since y =1, there exists unique vd such that– States rendered unobservable by input vd!

Note that exact tracking is too stringent– need initial cond.

But it’s easy to achieve exponential tracking– applying inputyields

How does tracking affect unobservable states ?

Tracking with stable inverse modelForward model:Dynamic inverse model:

Theorem: If forward and inverse models are (exponentially) stable, then feedforward input from dynamic inverse of internal model achieves (exponential) tracking for physical system.

– Trajectories converge for stable model pairs (M, M-1)– Feedforward input “asymptotically inverts” dynamics

Tracking with stable model pair (M, M-1)

Theorem implies:– For stable model pairs, trajectories x, x’ converge to – Feedforward input “asymptotically inverts” dynamics

(M, M-1)

Extensions and generalizationsForward model:Dynamic inverse model:

Results easily extend to accommodate:– multiple inputs / multiple outputs– (small) perturbations in dynamics

Sastry Springer 1999

– approximate input-output linearizationHauser PhD Thesis 1989; Hauser, Sastry, Kokotovic IEEE TAC 1992; Banaszuk, Hauser SIAM JCO 1996

– learning / adaptation / estimation of dynamicsSutton, Barto, Williams IEEE CSM 1992; Papavassiliou, Russell ICJAI 1999Sastry, Bodson Prentice Hall 1989; Vrabie, Vamvoudakis, Lewis IET 2013

Properties of dynamic inverse modelForward model:Dynamic inverse model:

Property: dynamic inverse model is unique– Exact tracking input determined by yd

– Independent of how internal model is represented or obtained (e.g. reinforcement learning, adaptive ctrl.)Sutton, Barto, Williams IEEE CSM 1992; Papavassiliou, Russell ICJAI 1999Sastry, Bodson Prentice Hall 1989; Vrabie, Vamvoudakis, Lewis IET 2013

– Impossible to learn if inverse model is unstable

Today: predictive models for interaction

Future: enhance human ability to interact with and control the built world– Cyber-Human Systems– Human Intranet– Cybathlon

Dynamics of humans embedded w/ machines

Humans are the enabling technology