Introduction Background Our Approach Results Summary

Learning Spatio-Temporally Encoded PatternTransformations in Structured Spiking Neural

Networks12

Andre Gruning, Brian Gardner and Ioana SporeaDepartment of Computer Science

University of SurreyGuildford, UK

9th November 2015

1http://dx.doi.org/10.6084/m9.figshare.15177792$Id: multilayerspiker.txt 1827 2015-11-09 07:39:53Z ag0015 $

Introduction Background Our Approach Results Summary

Introduction Background Our Approach Results Summary

Introduction Background Our Approach Results Summary

1 Introduction

2 Background

3 Our Approach

4 Results

5 Summary

Introduction Background Our Approach Results Summary

What are we doing?

What are we doing?

Formulate a supervised learning rule for spiking neuralnetworks that

can train spiking networks containing a hidden layer of neurons,

can map arbitrary spatio-temporal input into arbitrary outputspike patterns, ie multiple spike trains.

Why worthwhile?

Understand how spike-pattern based information processingtakes place in the brain.

A learning rule for spiking neural networks with technicalpotential.

Find a rule that is to spiking networks what is backprop torate neuron networks.

Human Brain Project

The SPIKEFRAME project:

“Structures of Learning Algorithms for Spiking NeuralNetworks”

joined HBP on 01/04/2015 through Competitive Callsprogramme

This work carried out as part of SP4 “TheoreticalNeuroscience”.

Introduction Background Our Approach Results Summary

Scientific Area

Where are we scientifically?

In the middle of nowhere between:

computational neuroscience

cognitive science

artificial intelligence / machine learning

Introduction Background Our Approach Results Summary

Introduction Background Our Approach Results Summary

1 Introduction

2 Background

3 Our Approach

4 Results

5 Summary

Introduction Background Our Approach Results Summary

Spiking Neurons

(a)

(c)

output spike

input spikes

output spike

input spikes

u

(b)

Spiking neurons: real neurons communicate with each other via sequences of pulses – spikes.

1 Dendritic tree, axon and cell body of a neuron.

2 Top: Spikes arrive from other neurons and its membrane potential rises. Bottom: incoming spikes onvarious dendrites elicit timed spikes responses as the output.

3 response of the membrane potential to incoming spikes. If the threshold θ is crossed, the membranepotential is reset to a low value, and a spike fired.

From Andre Gruning and Sander Bohte. Spiking neural networks: Principles and challenges.

In Proceedings of the 22nd European Symposium on Artificial Neural Networks, Computational Intelligence andMachine Learning – ESANN, Brugge, 2014.

Invited Contribution.

Introduction Background Our Approach Results Summary

Spiking Neurons

Spiking Information Processing

The precise timing of spikes generated by neurons conveysmeaningful information.

Synaptic plasticity forms the basis of learning.

Changes in synaptic strength depend on relative pre- andpostsynaptic spike times, and third signals.

Challenge: to relate such localised plasticity changes tolearning on the network level.

Introduction Background Our Approach Results Summary

Learning for Spiking NN

General Learning Algorithms for Spiking NN?

There is no general-purpose algorithm for spiking neuralnetworks.

Challenge: discontinuous nature of spiking events.

Various supervised learning algorithms exist, each with its ownlimitations eg:

network topology,

adaptability (e.g. reservoir computing),

limited spike encoding (e.g. latency, or spike vs no spike).

Most focus on classification rather than more challengingtasks like mapping from one spike train to another.

Introduction Background Our Approach Results Summary

Some Learning Algorithms for Spiking NN

SpikeProp 3, ReSuMe 4, Tempotron 5, Chronotron 6, SPAN 7,Urbanczik and Senn 8, Brea et al. 9, Freimaux et al. 10, . . .

3S.M. Bohte, J.N. Kok, and H. La Poutre. Spike-prop: error-backpropagation in multi-layer networks of spiking neurons.

Neurocomputing, 48(1–4):17–37, 20024

Filip Ponulak and Andrzej Kasinski. Supervised learning in spiking neural networks with ReSuMe: Sequence learning, classificationand spike shifting.Neural Computation, 22:467–510, 2010

5Robert Gutig and Haim Sompolinsky. The tempotron: a neuron that learns spike timing-based decisions.

Nature Neuroscience, 9(3), 2006.doi: 10.1038/nn1643

6Razvan V Florian. The chronotron: A neuron that learns to fire temporally precise spike patterns.

PLoS ONE, 7(8):e40233, 20127

A. Mohemmed, S. Schliebs, and N. Kasabov. SPAN: Spike pattern association neuron for learning spatio-temporal sequences.Int. J. Neural Systems, 2011

8R. Urbanczik and W. Senn. A gradient learning rule for the tempotron.

Neural Computation, 21:340–352, 20099

Johanni Brea, Walter Senn, and Jean-Pascal Pfister. Matching recall and storage in sequence learning with spiking neural networks.The Journal of Neuroscience, 33(23):9565–9575, 2013

10Nicolas Fremaux, Henning Sprekeler, and Wulfram Gerstner. Functional requirements for reward-modulated spile-timing-dependent

plasticity.The Journal of Neuroscience, 30(40):13326–13337, 10 2010

Introduction Background Our Approach Results Summary

Introduction Background Our Approach Results Summary

1 Introduction

2 Background

3 Our Approach

4 Results

5 Summary

Introduction Background Our Approach Results Summary

Our Approach

MultilayerSpiker

Generalise backpropagation to Spiking Neural Networks withhidden neurons.

Use stochastic neuron model

to connect smooth quantities (derivative exists) with discretespike trains (no derivative)

Introduction Background Our Approach Results Summary

Neuron model

Membrane potential

uo(t) :=∑

h

woh

∫ t

0Yh(t ′)ε(t−t ′)dt ′+

∫ t

0Zo(t ′)κ(t−t ′)dt ′ , (1)

o postsynaptic neurons, h presynaptic neuron

uo membrane potential of o.

woh strength of synaptic connection from h to o.

Yh(t) =∑

th<t δ(t − th) spike train of neuron h where th arethe firing times of h

Zo(t) =∑

to<t δ(t − to) spike train of neuron o where to arethe firing times of o.

Introduction Background Our Approach Results Summary

Neuron model

Spike response kernel ε and reset kernel κ

ε(s) = ε0 [e−s/τm−e−s/τs ] Θ(s) and κ(s) = κ0e−s/τm Θ(s) , (2)

spike response kernel ε0 = 4mV,

reset kernel κ0 = −15mV,

membrane time constant τm = 10ms,

the synaptic rise time τs = 5ms

Heaviside step function Θ(s).

Introduction Background Our Approach Results Summary

Neuron model

Stochastic Intensity (instantaneous firing rate) and Spikes

ρ(t) = ρ[u(t)] = ρ0 exp

(u(t)− ϑ

∆u

), (3)

firing rate at threshold ρ0 = 0.01ms−1.

“threshold” ϑ = 15mV.

smoothness of the threshold ∆uo = 0.2mV (output layer) or∆uh = 2mV (hidden layer)

Spikes are generated by a point process taking stochasticintensity ρo(t).

Ie in a small time interval [t, t + δt) a spike is generated withprobability ρo(t)δt.

Introduction Background Our Approach Results Summary

Backpropagation

Objective (“Error”) function

P(zrefo |x) = exp

(∫log (ρo(t))Z ref

o (t)− ρo(t)dt

), (4)

where Z refo (t) =

∑f δ(t − t f

o ) is the target output spike train forinput x .a

aJ. P. Pfister, T. Toyoizumi, K. Aihara, and W. Gerstner. Optimal spike-timing dependent plasticity for precise action potential firing

in supervised learning.Neural Computation, 18(6):1309–1339, 2006

Backprop approach

∆woh = ηo∂ logP(zref |x)

∂woh(5)

Introduction Background Our Approach Results Summary

Backprop approach

. . . and some ten slides later

Lots of derivatives, indices, probabilities.

Derivatives only possible due to smoothness of probabilityfunction.

Relatively freely switching between expected values and theirbest estimates to be had when you only have single cast.

Introduction Background Our Approach Results Summary

Backprop Weight Update

Backpropagated Error Signal

δo(t) :=1

∆uo

[Z ref

o (t)− ρo(t)], (6)

Hidden-to-Output Weights

∆woh = ηo

∫ T

0δo(t) (Yh ∗ ε)(t) dt . (7)

Input-to-Hidden Weights

∆whi =ηh

∆uh

∑o

woh

∫ T

0δo(t)([Yh(Xi ∗ ε)] ∗ ε)(t)dt . (8)

a

aBrian Gardner, Ioana Sporea, and Andre Gruning. Learning spatio-temporally encoded pattern transformations in structured spiking

neural networks.Neural Computation, To appear. 2015.Preprint available at http://arxiv.org/abs/1503.09129

Introduction Background Our Approach Results Summary

Introduction Background Our Approach Results Summary

1 Introduction

2 Background

3 Our Approach

4 Results

5 Summary

Introduction Background Our Approach Results Summary

Task

Task

Purpose: explore the properties of the new learning algorithm.

Map an input (given as a set of spike trains) to an output(given again as a set of spike trains).

Simulation details a.

aBrian Gardner, Ioana Sporea, and Andre Gruning. Learning spatio-temporally encoded pattern transformations in structured spiking

neural networks.Neural Computation, To appear. 2015.Preprint available at http://arxiv.org/abs/1503.09129

Introduction Background Our Approach Results Summary

Network Setup

0

20

40

60

80

100Input spike pattern

Xi

0

200

400

600

800

1000

Epis

odes

0 100 200 300 400 5000

200

400

600

800

1000

Time (ms)

Epis

odes

0 200 400 600 800 10000

1

2

3

4

Episodes

Dis

tance

E

D

A

B

C

Input

Hidden

Output

Output neuron

Hidden neuron

Left: spike rasters of input, hidden and output layers (withtargets).

Right top: network structure, bottom: van-Rossum distance.

Introduction Background Our Approach Results Summary

Network in Action

0

Xi

0 T0

(Xi ∗ ǫ)

0

uh − ϑ

0 T0

([Yh(Xi ∗ ǫ)] ∗ ǫ)

0

uo − ϑ

0 T

0

∆whi

Left: Input spike train Xi (top) and its evoked post synaptic potential Xi ∗ ε (bottom).

Middle: Fluctuations of a hidden neuron membrane potential uh relative to a firing threshold ϑ, inresponse to inputs from input layer (top). The potential dependent factor of the back propagated errorfrom hidden to input layer [Yh(Xi ∗ ∗ε)]

bottom left: corresponding PSP (according to kernel Xi ∗ ε).

Right: membrane potential of an output neuron uo , in response to hidden layer activity. Target indicatedby dotted lines (top). Weight changes of input-to-hidden weight due to learning rule.

Introduction Background Our Approach Results Summary

Experiments

0 8 16 24 32 400

20

40

60

80

100

Pe

rfo

rma

nce

(%

)

Input patterns

0 8 16 24 32 400

5000

10000

15000

Ep

iso

de

s

Input patterns

Free whi

Fixed whi

Single layer

Dependence of the performance on the number of input patterns andnetwork setup. Each input pattern mapped to a unique target output ofsingle output neuron and spike.

Left: performance as a function of the number of input patterns.

Right: Number of episodes to convergence in learning.

Blue curves: hidden weights whi updated according to learningalgorithm, red curves: fixed random weights (plus homoeostasis),green: single layer.

Introduction Background Our Approach Results Summary

Experiments

0 2 4 6 8 100

1

2

3

4

5

nh

/ n

o

Number of output spikes

no

= 10

0 0.5 1 1.5 2 2.5 30

20

40

60

80

100P

erf

orm

ance (

%)

nh

/ no

no

= 10

no

= 20

no

= 30

A B

Dependence of the performance on the ratio of hidden to output neurons,and the number of target output spikes. p = 50. A unique target outputspike pattern for each output neuron.

(Left) Performance as a function of the ratio of hidden to outputneurons.

(Right) Minimum ratio of hidden to output neurons required toachieve 90% performance.

Introduction Background Our Approach Results Summary

Introduction Background Our Approach Results Summary

1 Introduction

2 Background

3 Our Approach

4 Results

5 Summary

Introduction Background Our Approach Results Summary

Summary

Results

Compared to other learning algorithms for spiking neuronnetworks, we can learn

more input-output mappings: 20 classes or 200 individualpatterns here vs 3-4

more timed output spikes: up to 10 individually timed spikeshere vs 3-5

with multiple outputs: up to 30 here vs 1

Apply it!

MultilayerSpiker opens up the use of spiking neural networksfor technical/cognitive modelling tasks.

Spiking networks are biologically plausible.

Explore how computations can be done with neural networks.

Next step in the Human Brain Project: Implementation onSpiNNaker, and other neural hardware.

Introduction Background Our Approach Results Summary

Spiking Neural Networks

Open Questions

How do networks of spiking neurons carry out computations?

How can they learn such computations?

Does this explain how real biological neurons compute?

What is the applied killer application?