bayesia models of motor control encyclopedia

This article was originally published in the Encyclopedia of Neuroscience published by Elsevier, and the attached copy is provided by Elsevier for the

author's benefit and for the benefit of the author's institution, for non-commercial research and educational use including without limitation use in

instruction at your institution, sending it to specific colleagues who you know, and providing a copy to your institution’s administrator.

All other uses, reproduction and distribution, including without limitation commercial reprints, selling or licensing copies or access, or posting on open

internet sites, your personal or institution’s website or repository, are prohibited. For exceptions, permission may be sought for such use through

Elsevier's permissions site at:

http://www.elsevier.com/locate/permissionusematerial

Berniker M and Kording K (2009) Bayesian Models of Motor Control. In: Squire LR (ed.) Encyclopedia of Neuroscience, volume 2, pp. 127-133.

Oxford: Academic Press.

Bayesian Models of Motor Control 127

Author's personal copy

Bayesian Models of Motor Control

M Berniker and K Kording, Northwestern UniversityInstitute of Neuroscience, Chicago, IL, USA

ã 2009 Elsevier Ltd. All rights reserved.

Motor control is fundamental to the nervous system:only through our movements do we interact with theworld. Many models of motor control assume that ofall the ways we could move, we typically choose oneof the best. This implies that we can have a ‘notion’ ofhow bad one movement is in comparison to otherpossible movements. Within the framework of deci-sion theory, this notion is formalized by a cost func-tion that assigns a numerical value to every possiblemovement. The right movement decision seems sim-ple: choose the action a* that has the lowest asso-ciated cost. This decision process can be compactlydescribed by the equation,

a� ¼ argmina

ðcostðaÞÞ ½1�

For example, several studies have proposed that peo-ple move their hand from one position to another in afashion that is as ‘smooth’ as possible. Thus, the costfunction penalizes movements that are not smooth. Itwas found that many properties of human reachingmovements, such as the velocity profiles, can be wellpredicted by such cost functions.This framework assumes that we can deterministi-

cally choose a movement and be sure of its outcome.Yet, most motor outcomes are affected by uncertainty.To illustrate, if we aim at the center of a dartboard, weare not certain to hit the bullseye with the dart (even ifwe are a seasoned champion). If we continue to aim atthe center and throwmany times, wewill end upwith adistribution of dart positions. This distribution, p(x|a),characterizes the likelihood of each outcome, the dartposition x, given that we chose action a, the aimingpoint. This affects how we should choose the bestaction, the one that minimizes our likely cost:

a� ¼ argmina

Xpossibleoutcomes

cost ðoutcomeÞ pðoutcomejaÞ

0B@

1CA ½2�

In other words, we should choose the action that mini-

mizes our expected cost. In the example of playing darts,

the best aiming point is a point where we will receive high

scores even if we make large mistakes. Indeed, both ama-

teur andworld-class players are known to adopt a strategy

that is well predicted by this approach.

While playing darts, the errors largely stem frommotor noise (Figure 1(a)). However, there are many

Encyclopedia of Neuroscien

sensory sources of uncertainty as well. For example,our visual system is noisy and the location of thedartboard relative to our body is thus uncertain(Figure 1(a)). Moreover, our proprioceptive systemis noisy as well; the exact configuration of our bodyas we release the dart is uncertain (Figure 1(a)). Thesesources of uncertainty combine to produce uncer-tainty in the movement outcome given our movementdecision.

The difficulty in treating these movement decisionsis that the probability of an outcome given our actiondecision (p(x|a)) is generally difficult to compute. Theproblem is that computing this distribution requiresthe specification of information that is generallyuncertain. This includes information about our bodyas well as the external world. Bayesian inference spe-cifies how such pieces of information should be com-bined. It thus predicts how pieces of informationshould be integrated to allow for motor control thatminimizes our cost.

Bayesian Integration

Combining uncertain information to produce acoherent and accurate estimate of the world is acentral problem faced in motor control. A numberof psychophysics studies have analyzed how peopleintegrate multiple sources of uncertain information tomake sensorimotor decisions.

Combining Prior Knowledge with New Evidence

As an example of how we integrate uncertain infor-mation to estimate the world, consider the game oftennis. When playing tennis, it is helpful to estimatewhere the ball will land. Vision does not provideperfect information about the ball’s trajectory orvelocity, and we are thus uncertain where the ballwill land. The visual system, although noisy, stillprovides us with an estimate, or likelihood, of wherethe ball will strike the court. This likelihood is theprobability of having a particular sensory input foreach possible location the ball may land (Figure 1(b)).This knowledge may be combined with informationobtained from experience: the locations where theball may land are not uniformly distributed over thecourt. The locations are usually concentrated withinthe confines of the court and may be highly peakednear the baselines where it is most difficult to returnthe ball. This distribution of positions is called theprior (Figure 1(b)). Bayes’ rule defines how to combinethe prior and the likelihood to make an optimal esti-mate of the location where the ball lands (Figure 1(b)).

ce (2009), vol. 2, pp. 127-133

a

Likelihood

PriorPosterior

Posterior:p(x|o) = p(x)p(o|x)/p(o)

Prior:p(x)

Likelihood:p(o|x)

b

Posterior:p(x|o1,o2) = p(o1|x)p(o2|x)/p(o1)p(o2)

Cue 2:p(o2|x)

Cue 1:p(o1|x)

Prior–likelihood integration

Cue combination

c

Visual

Proprioceptive

Motor noise

x

Sensor noise

d

Figure 1 (a) Possible sources of noise in the motor system. Visual and proprioceptive senses contain inherent uncertainties. Motor

commands also possess noise and result in uncertain movements. (b) Bayesian integration in tennis: we view a ball traveling toward the

court floor and estimate the location where it will land, a likelihood (red). Experience frommany such instances tells us that the ball usually

lands close to the baseline, a prior (green). From these two pieces of information, we produce an updated estimate of where we believe

the ball will land, a posterior (blue ellipses). (c) Bayesian integration of a prior and likelihood. The prior, denoted with the green curve,

represents the probability of a state, x. The likelihood, denoted with the red curve, represents the probability of observing the data, o, given x.

The posterior is the probability that x is the state, given our observation, o. (d) Bayes’ rule applied to cue combinations.

128 Bayesian Models of Motor Control


Bayes’ rule states that the probability of the balllanding at position x given our observation o is theproduct of the prior probability of the ball at x, withthe likelihood of the observation o given the position x,divided by the probability of the observation:

pðxjoÞ ¼ pðxÞpðojxÞ=pðoÞ ½3�

The dist ribution produced by eqn [3] is shown gra-phically Figure 1(c). We can also interpret Bayes’ ruleas the ‘optimal’ means of combining a prior and alikelihood as it produces an estimatewith theminimumuncertainty. A few studies have shown that when peo-ple combine prior and new information, their behavioris close to that predicted by Bayes’ rule. Peoples’ abilityto integrate priors and likelihoods in a manner that isclose to the optimal prediction by Bayes’ rule has beenshown in many modalities.A typical study will let people use a motor task

to indicate their estimate of the location of a target.In every trial, this location will be drawn from aGaussian probability distribution (the prior). Onegroup of subjects experience a narrow target distribu-tion and thus have a narrow prior. A different groupof subjects will experience a wide target distribution,thus having a wide prior. Bayesian statistics predictshow subjects should combine the likelihood with the

Encyclopedia of Neuroscienc

prior. These predictions are then compared to humanperformance.

Such paradigms have been applied to numerousestimation problems. Sensorimotor integration, forceestimation, timing estimations, speed estimations, theinterpretation of visual scenes, and even cognitiveestimates are examples. These studies jointly demon-strate that people are very good at combining priorknowledge with new evidence in a way predicted byBayesian statistics.

Cue Combination

In many cases, it is not prior knowledge that is com-bined with a likelihood but, rather, two different cuesthat are combined into a joint estimate. For example,we may see and feel an object at the same time. Wecan then use what we saw and what we felt to inferthe properties of the object. Bayesian statistics allowsus to also solve these problems and the posterior issimply the product of the likelihoods (Figure 1(d)).Some studies have found that people are also optimalat this combination.

As a typical study, we consider an analysis of howpeople combine visual and auditory information toestimate the position of a target. In such a study,the precision of visual and auditory perception is

e (2009), vol. 2, pp. 127-133



measured for each subject (unimodal). Subsequently,the precision and accuracy of perception is measuredwhen subjects can use both modalities (bimodal). Theperformance in the bimodal trials can be predictedfrom the unimodal trials using Bayesian statistics.As with studies investigating prior–likelihood inte-

gration, such a paradigm has been applied to numer-ous cue combination problems. It has been applied tothe combination of visual and auditory informationas well as tactile and visual information. It has alsobeen applied to several cases in which two pieces ofknowledge from the same modality are combined.For example, texture and stereo disparity are bothcues for estimating the depth of a visual object. Inall tested cases, cues are combined by the subjects in amanner that is close to the optimum prescribed byBayesian statistics.

Bayesian Integration over Time

The state of the world and our information about itare continually changing. We thus constantly need tointegrate this new information with our currentbeliefs to inform our estimates of the world. Thisimplies that Bayesian integration should take placein a continuous manner.This approach is extensively taken in modern appli-

cations of control theory. Many of these applicationsare built on the idea of Kalman filtering, which is amethod for using an error in our prediction of theworld to continuously update our beliefs (Figure 2(a)).This technique, although not usually described assuch, is equivalent to a sophisticated application ofBayesian statistics. At every time step, a Kalman filtercombines its estimate of the world’s state (the prior)with a measured observation (the likelihood) toupdate the prediction of the world’s current state,

Model Bayesianupdate

Observation(likelihood)

Motorcommand

Predictedobservation

(prior)

Updatedprediction(posterior)

Δt

Sta

te

a b

Figure 2 (a) Typical procedure for optimally estimating the world’s sta

motor command, is used to estimate a predicted state, or prior distrib

particular observation given the current world state (red). A Kalman fi

current state (blue). (b) This process of Bayesian inference repeats it

analogous update procedure. Our prior belief of muscle strength (gre

belief of our muscle’s strength.


represented by a posterior (Figure 2(b)). This formal-ism is well developed and used in a wide range ofapplications from aeronautics to humanoid robotics.Indeed, even themotor control problems of two appli-cations as disparate as controlling a jet and controllingour bodies share many computational analogies. Inboth cases, continuously incoming information needsto be assessed to move precisely (albeit on differenttimescales). Only recently has this methodology ofKalman filtering been applied to make quantitativepredictions of human movement behavior.

A number of studies have examined how subjectsintegrate information in a continuous fashion. Thesestudies have compared the way in which subjectsmove with the predictions of optimal Bayesian mod-els. They have found close agreement between thepredictions of the optimal theory and the way move-ments are actually executed.

As an example of this problem, we consider a recenttheory of motor adaptation. The properties of ourmuscles change continuously throughout our lives.Over several timescales, muscles change – fatigueover the course of hours, disease over the course ofweeks, and aging overmany years. Thismeans that wemust estimate the strength of our muscles if we are tomove precisely. The error in our estimate of thestrength of ourmuscles will translate into amovementerror. We can use this error to obtain a likelihoodfunction characterizing the strength of the muscles.According to Bayesian statistics, we need to combinethe newly obtained information, ourmotor error, withour prior belief.We can thus infer a new and improvedestimate (Figure 2(c)). Often, adaptation is probedusing a target jump paradigm. In these paradigms,an experimental manipulation is used to changesubjects’ perception of the strength of their limb. Inthese circumstances, Bayesian statistics predicts that

c

Target

}

Strength

Tim

e

Error

te in modern control theory. A model of the world, combined with a

ution (green). Observations of the world dictate the likelihood of a

lter is used to make a Bayesian update of our belief in the world’s

self at each time step. (c) Motor adaptation can be framed as an

en) is integrated with our observed motor error (red) to update a

ce (2009), vol. 2, pp. 127-133



subjects should adapt over time. The way subjectsadapt in this setting is then compared to the adapta-tion profile predicted by optimal Bayesian adaptation.It has been found that motor adaptation over time

can be well understood from the predictions ofKalman filtering. It has also been found that theway people estimate the position of their hand in thedark is well predicted by the same methods. Asanother example, when people balance a pole ontheir hand, they also seem to use such strategies.These studies highlight that when people integrateinformation over time, they seem to do so in a mannerthat is consistent with the optimal Bayesian solution.

Bayesian Inference of Structure

We previously discussed how people combine twocues into a joint estimate using Bayes’ rule. The appli-cation of these models implicitly assumed that we arecertain that both cues have a common source: if wehear a tone while seeing a flash, we assume that bothof the cues originate at the same position and have acommon cause. This idea has been used very success-fully for describing human behavior in cue combina-tion experiments.Although successful as implemented thus far, these

probabilistic models have obvious shortcomings. Forinstance, we know from experience and experimentalstudies that if two cues are very dissimilar our percep-tion of unity breaks down. If we see a flash far to ourright while perceiving a tone far to our left, we per-ceive two independent events. We infer the structureof events; we estimate if two events have a commoncause or if they just randomly co-occur. This impliesthat if two events are perceived as having a commoncause, they should influence the perception of eachother. On the other hand, if they are perceived as nothaving a common cause, the perception of eachshould be independent of the other. In some circum-stances, the cues can create an illusion of a commoncause, the so-called ventriloquist effect.Traditional Bayesian models of psychophysical

behavior are unable to explain this range of phenom-ena. Indeed, experiments that tested these modelsused cues that were close to coincident. For example,the position of a visual stimulus was close to theposition of an auditory stimulus.Other studies have tested subject performance in

situations in which two cues are dissimilar from oneanother. A typical study will present a visual stimulusat a random position within the visual field andsimultaneously present an auditory stimulus at a dif-ferent random position. Subsequently, subjects areasked where they perceive the stimuli.


These studies have found that when two stimuli arenear coincident, people tend to infer a common causeand use each cue to guide estimation of the other.With increasing disparity between the cues, subjects’belief in a common cause decreases (Figure 3). Tradi-tional Bayesian models predict that in the case ofGaussian likelihoods the estimate of the value of onecue should be linearly influenced by the value of theother cue. In sharp contrast to these predictions,experiments have found nonlinear cue interactions.These interactions are well predicted if we assumethat on each trial the subjects infer a common ordistinct cause for both cues.

These effects are found in a wide range of experi-mental situations. They have been found in positionestimation using visual and auditory stimuli, in theestimation of depth from several cues, and also innumber estimation using visual, tactile, and auditorystimuli. Evidence suggests that subjects integrate notonly priors and likelihoods but also the causal struc-ture underlying a set of sensory stimuli.

The statistical problem that subjects solve in cuecombination implicitly involves an inference aboutthe causal structure of the stimuli. In these studies,people are uncertain about the identity and number ofrelevant variables; for example, did the flash of lightcause the auditory tone, did the tone cause the flash,was there an unknown cause for them both, or werethey simply coincidental? The problem faced by thenervous system is thus similar to cognitive problemsthat occur in the context of causal induction. Manyexperiments demonstrate that people interpret eventsin terms of cause and effect. The results presentedhere show that sensorimotor integration exhibitssome of the factors that make human cognition diffi-cult to understand. Carefully studying and analyzingseemingly simple problems such as cue combinationmay provide a fascinating way of studying the humancognitive system in a quantitative manner.

Future Directions for Bayesian Inference

The Bayesian models reviewed so far do well at pre-dicting the results from many experimental studies ofmotor control. Whereas in theory Bayesian statistics’applicability is rarely limited, in practice there aremany cases in which this technique is hindered.

Most of the previously discussed Bayesian modelsassume that noise sources are Gaussian and that theinteractions between the subject and the world arelinear. Recent approaches in Bayesian statistics allowthe application of powerful framework well beyondthese simple assumptions. For instance, the structureinference problem is a simple case of non-Gaussian

e (2009), vol. 2, pp. 127-133

xvis xaud

x

a

x1

xvis xaud

x2

Common cause

Independentcauses

Visualnoise

Visualnoise

Auditorynoise

Auditorynoise

b

Cue disparity

p(~

com

mon

)

p(co

mm

on)

Figure 3 (a) Two structural beliefs of the world. If two cues are adequately coincident, subjects perceive them as having a common

cause (green box), a phenomenon typified through ventriloquism. If the cues are disparate in time or space, subjects perceive them as

having independent causes (red box). (b) Subjects’ belief of common cause. Because the spatial disparity of two cues, a light flash and a

tone, is experimentally controlled, the belief in a common cause can be manipulated.



proba bility distri butions. The techni ques that may beapplied to such pro blems range from linearizat iontechniq ues such as extende d Kal man filters ov er vari-ation al techniques to particl e-filter ing method s. Thespecial ized Bayesian liter ature deals extens ively withsuch compu tational difficult ies.Ba yesian infe rence allows us to estim ate the pr esent

state of the world given all the sensor y obs ervationswe have obtai ned from the pa st until now. If theproble m we are faced with requires making only onedecision at a single point in time (static problem),then decision theory (see eq n [2]) readi ly allows usto decide optimally. Yet, movement is fundamentallya sequential decision (dynamic) problem. Any motorcommand we produce in the present influences thesituations in which we will find ourselves in thefuture. This implies that motor commands are notindependent from each other but, rather, each com-mand must be viewed in the context of all possiblefuture situations it may direct us.


Optimal control theory is the systematic study ofproblems of this class. Often utilizing a Bayesianframework, it employs analytical and numerical tech-niques to solve the motor control problem. For the setof problems in which dynamics are linear, noise isGaussian, and cost functions are quadratic, optimalcontrol provides efficient solutions. Much researchaddresses ways of extending the approaches to morecomplicated problems. Typical applications involvethe control of humanoid robots and the control ofaircraft. The theory is complex and beyond the scopeof this article. Advances in optimal control coupledwith advances in Bayesian methods hold promise forallowing progressively more complicated movementbehaviors to be analyzed and predicted.

Discussion

Much if not most of neuroscience is driven by postu-lates regarding the problems the nervous system is

ce (2009), vol. 2, pp. 127-133



attempting to solve. For example, researchers record-ing from the visual system may assume that visualfeatures such as edges are of importance to the objectrecognition problem. When researchers study themotor system, they may decide that efficient move-ments are the intended goal. This way of thinking iscalled normative: a description of the problem beingsolved is converted into predictions of how the ner-vous system ‘should’ work. Models that predictbehavior from such descriptions are especially infor-mative and make precise quantitative predictions ofexperimental data.Normative models prescribe how the nervous sys-

tem should behave, as opposed to neural modeling,which describes how the nervous system behaves. Toclarify the distinction, models derived from meaning-ful descriptions of the problem to be solved should beconsidered normative or prescriptive. For example,some theories of movement assume that people tryto move as smoothly as possible, and these modelsexplain a large amount of experimental data. How-ever, there is no explicit reason why smooth move-ments would be better than similar but less smoothmovements. Perhaps more to the point, these modelsdo not yield insight on why subjects move the waythey do. In the case of smooth movements, researchdemonstrated that a different cost function, clearlytied to the movement problem, could also predict thedata; optimizing movements for precision naturallyleads to smoothmovements – a cost function that is ofobvious importance.A different way of modeling the properties of the

nervous system is the so-called ‘bottom-up’ approach.Numerous measured or inferred properties of neu-rons are used to run comprehensive computer simula-tions of neurons within expansive networks, theresults of which are meant to mimic the actual ner-vous system. This approach relies on the accurateknowledge of many system parameters – a formidablerequirement. Furthermore, in large, complex systemssuch as these, small deviations in parameters canresult in gross and qualitative differences from theactual system being emulated – in this case the ner-vous system. Even in areas where bottom-up model-ing is practical, such as action potential propagationin squid giant axons, normative models may contrib-ute to an understanding of the system. It has beenconjectured that the density of sodium channels isoptimally adapted to maximize the speed of actionpotential conductance, very possibly the result ofevolutionary pressures to minimize the flight responsetime. This is of obvious normative reasoning becausefleeing fast from approaching predators can meanthe difference between life and death. Normative


modeling of low-level cellular properties is possibleand in close analogy with the normative ‘top-down’modeling of behaviors made by decision theory.

Bayesian integration offers a computational frame-work for combining various sensory information,from multiple modalities and different points intime, into a joint estimate of the world. Consistentwith these ideas, unimodal brain regions are knownto be affected by uncertainty. For example, primaryvisual cortex is influenced by the statistics of a stimu-lus. Multimodal regions that combine informationare similarly influenced by probabilities. Prefrontal,temporal, and parietal cortices are likely to exhibitsimilar statistical principles. Crucial in the timing ofmotor behaviors and motor learning, the cerebellum’sfunction can also be explained in Bayesian terms.Interestingly, however, even cortical regions tradi-tionally thought of as unimodal, such as primaryvisual and auditory cortices, have been shown to beresponsive to multiple sensory modalities and receiveprojections from multiple sensory regions. Takentogether, the functional and anatomical connectivityof the brain is consistent with the multimodal com-putations necessary for Bayesian inference. Consider-ing how salient the problem of uncertainty is,numerous, if not all, brain regions can be expectedto utilize Bayesian statistics to reduce uncertainty.

Bayesian statistics and normative modeling holdpromise for many areas of neuroscience because theyformalize the problems the nervous system needs tosolve while addressing the inherent uncertainties.Normative models have few free parameters in com-parison to neural models – radically simplifying theirapplication. Moreover, normative modeling is syner-gistic with neural modeling; combining the two tech-niques will provide models that not only predict butalso explain neural behavior. By quantitatively pre-dicting behavior from these normative–neural mod-els, we can develop a deeper understanding of theproblems being solved by the nervous system. Thiswill also allow us to ask better, more informed ques-tions in future neuroscientific experiments.

See also: Bayesian Cortical Models; Computational

Approaches to Motor Control; Motor Psychophysics;

Sensorimotor Integration: Models.

Further Reading

Davidson PR and Wolpert DM (2003) Motor learning and predic-tion in a variable environment. Current Opinion in Neurobiol-ogy 13(2): 232–237.

Ernst MO (2005) A Bayesian view on multimodal cue integration.In: Knoblich G, Thornton I, Grosjean M, and Shiffrar M (eds.)

e (2009), vol. 2, pp. 127-133



Human Body Perception from the inside Out, pp. 105–131.New York: Oxford University Press.

Gold JI and Shadlen MN (2002) Banburismus and the brain:

Decoding the relationship between sensory stimuli, decisions,and reward. Neuron 36: 299–308.

JordanMI (1998) Learning in Graphical Models. Cambridge, MA:

MIT Press.Kersten D and Yuille A (2003) Bayesian models of object percep-

tion. Current Opinion in Neurobiology 13(2): 150–158.

Knill D and Richards W (eds.) (1996) Perception as BayesianInference. New York: Cambridge University Press.

Kording KP and Tenenbaum JB (2006) Causal inference in sensori-

motor integration. Advances in Neural Information ProcessingSystems 19.

Kording KP andWolpert DM (2006) Bayesian decision theory in sen-sorimotor control. Trends in Cognitive Sciences 10(7): 320–326.

MacKay DJC (2003) Information Theory, Inference, and LearningAlgorithms. New York: Cambridge University Press.


Maloney LT, Trommershauser J, and Landy MS (2006) Questions

without words: A comparison between decision making under

risk and movement planning under risk. In: Gray W (ed.)

Integrated Models of Cognitive Systems. New York: OxfordUniversity Press.

Niv Y, Joel D, and Dayan P (2006) A normative perspective on

motivation. Trends in Cognitive Sciences 10(8): 375–381.Saunders JA and Knill DC (2004) Visual feedback control of hand

movements. Journal of Neuroscience 24(13): 3223–3234.Scott SH (2004) Optimal feedback control and the neural basis of

volitional motor control. Nature Reviews Neuroscience 5(7):532–546.

Shams L, Kamitani Y, and Shimojo S (2000) Illusions.What you see

is what you hear. Nature 408(6814): 788.Todorov E (2004) Optimality principles in sensorimotor control.

Nature Neuroscience 7(9): 907–915.Todorov E (2006) Optimal control theory. In: Doya K (ed.)

Bayesian Brain. Cambridge, MA: MIT Press.

ce (2009), vol. 2, pp. 127-133

bayesia models of motor control encyclopedia

Documents