Progress in MURI 15 ( ) Mathematical modeling of decision behavior. AFOSR, Alexandria, VA, Nov 17th, 2009 Phil Holmes 1. Optimizing monkeys? Balancing noisy stimulus information and reward expectations (Feng, Holmes, Rorie, Newsome, PLoS Comp. Biol. 5 (2), 2009). 2. Time-dependent perturbations can distinguish integrator dynamics (Zhou, Wong-Lin, Holmes, Neural Comput., 21, , 2009). [3. Mean field reduction of a spiking neuron integrator model with neuromodulation (Eckhoff, Wong-Lin, Holmes, SIAM J. Appl. Dyn. Sys., in review, 2009). Not to be presented today. ] 1. Optimizing Monkeys & Optimistic Math Leaky competing accumulators reduce to OU (actually, nonlinear DD/OU processes) via strong attraction to a 1-d subspace: (Usher & McClelland, 1995,2001) Subtracting the accumulated evidence yields an OU process (linearized DD case shown here). This generalizes to nonlinear systems: stochastic center manifolds. Fixed viewing time (cued response) tasks can be modeled by DD/OU processes. We consider the PDF of sample paths of the SDE, which is governed by the forward Fokker-Planck or Kolmogorov PDE: Interrogate solution at cue time T : is ? [Neyman-Pearson] General solutions for time-varying drift (SNR) are available, so we can predict accuracy (psychometric functions): Integrating bottom-up (stimulus) and top-down (prior expectation) inputs: a motion discrimination task with four reward conditions. Introduces reward priors (expectations). A. Rorie & W.T. Newsome, SfN 2006; in prep., 2008). DD can incorporate priors: reward expectations Reward cue Motion We only model this part HL HH LL LH Why monkeys? Electrophysiology! (Rorie, Reppas, Newsome) DD/OU models with reward bias priors, 1 Choose a model for reward expectation. Two examples are: 1. Initial condition set at start of motion period: motion period only. 2. Bias applied to drift rate throughout reward cue and motion periods: reward cue period motion period. for predicts shifted PMFs: Feng, H, Rorie, Newsome, PLoS Comp. Biol. 5 (2), Leak-dominatedBalanced Inhibition- dominated (stable OU, recency) (DD, optimal) (unstable OU, primacy) HL LH HH LL DD/OU models with reward bias priors, 2 These reward expectation models both lead to PMFs that can be written in the simple form where b 1 acuity = ability to use signal (SNR) b 2 shift = estimate of reward bias depending on the model chosen. But unless we have a range of priming and viewing times*, the models can t be distinguished on behavioral evidence alone : the parameters cannot be separated. [*But see McClelland presentation.] We can nonetheless compute shifts that maximize expected rewards, e.g., in simple case of fixed coherence : (More complex expressions hold for the mix of coherences actually used.) ~ 1/SNR reward ratio DD/OU models with reward bias priors, 3 Fit the model to the Rorie-Newsome behavioral data from two adult male rhesus monkeys, averaged over all sessions: monkey A monkey T slope shift For fixed the PMFs all reduce to a 2 parameter family: DD/OU models with reward bias priors, 4 Maximize expected rewards: compute optimal shifts given animals slopes and compare with their actual shifts: Both animals overshift, and T prefers alternative 2 when rewards are equal. But does this cost them much? How steep is the reward hill? reward ratio sums of Gaussians DD/OU models with reward bias priors, 5 Expected reward functions are rather flat, and overshifting costs less than undershifting. So they don t lose much! monkey A monkey T 99.5% performance bands DD/OU models with reward bias priors, 6 Overall performance is good, although acuity (= slope b 1 ) and shifts (b 2 ) vary significantly from session to session. In spite of visual correlation, the animals do not exhibit significant b 1 vs b 2 correlations (performance tuning). Monkey A: Blue is optimal; magenta curves are 99% & 97% of optimal. With few exceptions, A stays inside 97% in every session. Monkey T is not as good, shows bias to T 2. DD processes extend to include top-down cognitive control: can be used to predict optimal PMFs. Two subjects tested both overshift, but garner 98 99% of maximum possible rewards, in spite of significant session-to-session variability. App of rigorous math methods. Brief impulsive perturbations during stimulus presentation can reveal the dynamics of integrators, distinguish pure DD processes from leak- and inhibition-dominated integrators*, and even from nonlinear model. (*stable, unstable OU processes) Conclusions 1 2. Brief perturbations can distinguishing integrators Drift-diffusion and OU processes with thresholds can model RT tasks. Zhou, Wong-Lin, H, Neural Comput. 21 (8), 2009. Time-dependent perturbations can distinguish integrator types. Decision variable x time threshold for choice 1 threshold for choice 2 Mean RT Variance Perturbation modifies RT distributions means standard deviations Means and standard deviations of RT distributions are shifted in distinct manners for constant and rising drift rates (CD, TD) and for stable and unstable OU processes (SOU, UOU); positive pulse +, negative pulse _. Zhou, Wong-Lin, H, Neural Comput. 21 (8), Perturbn must be early to have strong effect! Zero-effect perturbations CD, TD, SOU and UOU are most easily distinguished by tuning a neg-pos pulse pair so that the mean RT is not changed. These employ the recency/primacy effects previously noted by Usher-McClelland. The simple patterns seen here occur only for low noise and early perturbations (pert time T