encoding & decoding of neuronal ensembles in motor cortex nicholas hatsopoulos dept. of...

Encoding & Decoding of Neuronal Ensembles in Motor Cortex

Nicholas HatsopoulosDept. of Organismal Biology & Anatomy

Committees on Computational Neuroscience & NeurobiologyUniversity of Chicago

Co-founder & board member of Cyberkinetics, Inc.

Encoding Problem

trial 1trial 2trial 3trial 4trial 5

Decoding Problem

Behavior

Multi-trial averaging

Single-trial prediction

“The motor cortex appears to be par excellence a synthetic organ for motor acts… the motor cortex seems to possess, or to be in touch with, the small localized movements as separable units, and to supply great numbers of connecting processes between these, so as to associate them together in extremely varied combinations. The acquirement of skilled movements, though certainly a process involving far wider areas (cf. V. Monakow) of the cortex than the excitable zone itself, may be presumed to find in the motor cortex an organ whose synthetic properties are part of the physiological basis that renders that acquirement possible.”

Leyton & Sherrington (1917)

The two components of language

• Words or elementary primitives of meaning

• Rules or grammar by which the primitives are combined

Pinker (1999)

The language of motor action in motor cortex

• Motor primitives: position, velocity, direction, trajectory

• Motor grammar: addition

Center-Out Task

Directional Tuning

time (s)

0°

45°

90°

135°

180°

225°

270°

315°

20 40

frequency (Hz)150

0-0.5 1.00

Behavioral Apparatus

Random-walk task

B0 20 40 60 80 100 120

140

160

180

200

220

240

260

Utah/Bionic Technologies ProbeRichard Normann U Utah

Chronic Multi-electrode array

Primary Motor Cortex (MI)

LegLeg

FaceFace

ArmArm

5 mm5 mm

MIMI

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0 100 200 300 400 500 600 700 800 900 1000

DAYS POSTIMPLANT

AR

RA

Y Y

IEL

D

subject 1 (tom)subject 2 (coco)subject 3 (buddy)subject 4 (radley)

Long-term Reliability & Stability

Many Neurons Every Day(19 tests over 110 days)Blue - no recordingRed - best recordings

Why view motor cortical encoding astime-dependent?

• Trajectory-selective activity in motor cortex (Hocherman & Wise, 1990, 1991)

• Preferred directions shift in time (Mason et al., 1998; Sergio et al., 2005; Sergio & Kalaska, 1998)

-200 0 200

0 200 400 600

-200 0 200

Center-out taskShifts versus time

-200 0 200

0 200 400 600

-200 0 200

Center-out task Random-walk taskShifts versus time Shifts versus lead/lag time

movement onset

lag lead

Temporal tuning (information theory)

Lead/lag (s)

The Encoding Model

A class of general linear models (e.g. logistic regression) that estimates the probability of a spike given a particular movement trajectory:

γvke)v|tP(spike

)()...(),(),...1(),(),1(),...,( tvtvtvtvtvtvv yxxxxx

k

= preferred velocity trajectory

= preferred trajectory and path (“pathlet”)k

integrated

γycxbsav

vk

e)y,x,s,v|tP(spike

)(

X position (mm)

20 40 60 80 10 0 1 201 40

1 60

1 80

2 00

2 20

2 40

2 60

0

0

t = to

γycxbsav

vk

model input =

model input

pro

ba

bili

ty o

f a

sp

ike

lead/lag (ms)

-3.5 -3 -2.5

0.040.06

0.080.1

0.12

-4 -3 -2

0.05

0.1

0.15

0.2

-6 -4

0.02

0.04

0.06

-100 0 100200300

00.2

0.4

0.6

0.8

-100 0 100 200300-0.5

0

0.5

1

-100 0 100200 300

0

0.5

1

X po s itio n-2 0 2 4

-2

0

2

4

-10 -5 0

0

5

10

-5 0 5 10 15-15

-10

-5

0

5

Neuron 1 Neuron 3 Neuron 6 Neuron 8 Neuron 9 Neuron 15 Neuron 19 Neuron 20 Neuron 27

Neuron 29 Neuron 30 Neuron 31 Neuron 32 Neuron 36 Neuron 37 Neuron 38 Neuron 39 Neuron 40

Neuron 41 Neuron 45 Neuron 47 Neuron 48 Neuron 49 Neuron 50 Neuron 51 Neuron 52 Neuron 53

Temporal stability of pathlet representation

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

False alarm probability

Hit

prob

abili

ty

red +/-50 msblue +/-150 msmagenta +/-300 mscyan +/- 500 ms

ROC analysis to quantify the performance of the encoding model

Lead/lag time (ms)-300 -200 -100 0 100 200 300

0.37

0.38

0.39

0.4

0.41

0.42

Encoding Performance as a function of trajectory length

-300 -200 -100 0 100 200 30035

40

45

50

55

60

65

Lead/lag time (ms)

Decoding Performance as a function of trajectory length

C audal

R ostral

X po s itio n

rs041130; MI

Map of Pathlets

0 0.5 1 1.5 2 2.5 3-0.05

0

0.05

0.1

0.15

Inter-electrode distance (mm)

Horizontal connectivity in motor cortex

1 mm Huntley & Jones (1991)

Rule for combining pathlets: Additive rule

Assuming conditional independence,

)|()|(& 2121 vspPvspP)v|spP(sp

22111212 vkvkγvk eee

2112 kkk

-4 -2 00

0.2

0.4

model input

spik

e p

rob

ab

ility neuron 1

0 5 10-10

-5

0

x-position

y-p

osi

tion

integrated k1

-3 -2 -1 00

0.2

0.4

neuron 2

-5 0 5

-10

-5

0

integrated k2

-6 -4 -20

0.05

0.1

0.15joint spiking model

-5 0 5 10 15-20

-10

0

integrated k12

-6 -4 -20

0.05

0.1

product of singleneuron models

-5 0 5 10 15-20

-10

0

integrated (k1+k

2)

r1031206; MI; 47&48:+/-5 ms

-8 -6 -4 -20

0.05

0.1

0.15neuron 1

model input

spik

e p

rob

ab

ility

-30 -20 -10 0-20

0

20

x-position

y-p

osi

tion

integrated k1

-3 -2 -1 00

0.2

0.4

neuron 2

-5 0 5

-10

-5

0

integrated k2

-10 -50

0.01

0.02joint spiking model

-30 -20 -10 0-30

-20

-10

0

10

integrated k12

-10 -50

0.01

0.02

product of singleneuron models

-30 -20 -10 0-30

-20

-10

0

10

integrated (k1+k

2)

r1031206; MI; 15&48;+/-5ms

-4 -3 -2 -10

0.1

0.2neuron 1

model input

spik

e p

rob

ab

ility

-5 0 5-5

0

5

x-position

y-p

osi

tion

integrated k1

-3 -2 -1 00

0.2

0.4

neuron 2

-5 0 5

-10

-5

0

integrated k2

-6 -5 -4 -30

0.02

0.04joint spiking model

-5 0 5-10

-5

0

integrated k12

-6 -5 -4 -30

0.02

0.04

product of singleneuron models

-5 0 5-10

-5

0

integrated (k1+k

2)

r1031206; MI; 2 vs. 48;+/-5ms

neuron 39 vs. neuron 51

-0.4 -0.2 0 0.2 0.4

20

40

60

80

100

neuron 40 vs. neuron 41

-0.4 -0.2 0 0.2 0.4

20

60

100

140

180

220

260

Cou

nts/

bin1 ms bin

Case #1 Case #2

Potential violations of conditional independence

Spike Jitter Method

neuron 1(reference)

+J-J

+/-w

2 parameters:

w, time resolution of synchrony J, the jitter window

neuron 2(target)

2 spikejitteredtrains

10% of all cell pairs (N=1431) show significant synchrony at a resolution of +/-5 ms, p<0.05

Case #1 Case #2 1000 jitters

Potential violations of conditional independenceCase #1

Case #2

-20 -10 00

0.02

0.04

0.06

-14 -12 -10 -8 -6 -4 -20

0.05

0.1

0.15

-1 2 -1 0 -8 -6 -4 -21 0

1 0

1 0

-1 4 -12 -1 0 -8 -6 -4 -21 0

1 0

1 0

21212121 )()(21 )&( vkkoffsetvkk egespspP

Conclusion: Synchronization preserves additive rule but increases the sensitivity of the tuning function by increasing the gain

When conditional independence appears to be violated

Computer

Neuro-motor prosthetic system

1) Multi-electrode array implant

2) Decoding of neural signals

3) Output interface

Personal computer:• mouse• keyboard

Assistive Robotics:• robotic arm• mechanized prosthetic arm

Biological interface:• muscles • peripheral nerve • spinal cord

Sensor

Cable

Cart

BrainGateTM Pilot Device

BrainGateTM Sensor Implantation and Post-Op Recovery as Planned

• Surgery as planned

• Post-op recovery unremarkable

• Wound healing around pedestal complete

Arrayon Cortex

Insertion

2 months post implant

Binary Modulation-Imagined Opening/Closing of Hand

BrainGate Signal Detection and Analysis;SCI Able to Modulate Neural Output

XRRRf T1T

Warland et al. (1997)

2. Algorithmic level:Optimal Linear Filter Reconstruction

RfX(t)ˆ

Estimated position of hand in time

Response of neural ensemble in time

filter coefficients

Two Dimensional Cursor control

Hatsopoulos LabQingqing XuWei Wu, PhD

Sunday Francis Zach HagaJignesh JoshiJohn O’LearyDawn PaulsenJake ReimerJonathan KoJoana Pellerano

Richard Penn, MDMatt FellowsYali AmitCyberkinetics, Inc.

Cross-Validated Performance as a function of pathlet length

Cross-validated Pathlet Population Vector Decoding