SPICE workshop, Mainz 2019

Antiferromagnet/ferromagnet

heterostructures as

synapses and neurons

A. Kurenkov1-3, S. DuttaGupta1-3, C. Zhang1,4,5

S. Fukami1-3,5,6, Y. Horio1 and H. Ohno1-3,5,6

1 Laboratory for Nanoelectronics and Spintronics, RIEC,

2 CSIS, 3 CSRN, 4 FRIS, 5 CIES, 6 WPI-AIMR (Tohoku University)

Spiking neural networks with CMOS

CMOS-based

synapse

CMOS-based

neuron

E. Chicca et al., Proc. IEEE 102, 9 (2014)

1 million neurons

PNAS 105, 3593 (2008)

0.001% of the human brain

1/60th of its speed

3 GHz CPU x60

100 billion neurons at full speed

=

60 × 105 × 60

=

36 million CPUs /

1011 neurons and 1015 synapses

≈

1016 transistors

≈

2 million CPUs /

2.5 million neurons

Science 338, 1202 (2012)

1 million neurons

Science 345, 6197 (2014)

“True North” by IBM

8 million neurons

IEEE Micro, 38, 1 (2018)

“Loihi” by Intel

1 million neurons

PNAS 105, 3593 (2008)

0.001% of the human brain

1/60th of its speed

3 GHz CPU x60

1995 2000 2005 2010 2015 202010

0

101

102

103

104

105

106

107

108

109

1010

1011

1012

CMOS CPU

"Performance" SNN

"Exploratory" SNN

Tra

nsis

tors

/

N

eu

ron

s

Year

Human

Pigeon

Spiking neural networks with CMOS

Why bother with number of neurons?

“Overall brain size … best predicts cognitive ability across non-human primates”

R. O. Deaner et al., Brain Behav. Evol. 70, 115 (2007)

What properties are we seeking?

Leaky integrate-and-fire

Spike-timing-dependent plasticity

New material approaches

Leaky integrate-and-fire

Spike-timing-dependent plasticity

S. Boyn et al., Nat. Commun. 8, 14736 (2017)

Ferroelectric cell

C. Zamarreno-Rames et al., Front. Neurosci. 5, 26 (2011)

Leaky integrate-and-fire

Spike-timing-dependent plasticity

S. Boyn et al., Nat. Commun. 8, 14736 (2017)

Ferroelectric cell

Mott insulatorP. Stoliar et al., Adv. Funct. Mater. 27, 11 (2017)

New material approaches

Leaky integrate-and-fire

Spike-timing-dependent plasticity

S. Boyn et al., Nat. Commun. 8, 14736 (2017)

[ Ferroelectric tunnel junction ]

P. Krzysteczko et al., Adv. Mater. 24, 762 (2012)

[ Magnetic tunnel junction ]

Y.-F. Wang et al., Sci. Rep. 5, 10150 (2015)

[ Resistive ]

D. Kuzum et al., Nano Lett. 12, 2179 (2012)

Y. Li et al., Sci. Rep.3, 1619 (2013)

[ Phase-change ]

F. Alibart et al., Adv. Funct. Mater. 22, 609 (2012)

[ Hybrid organic-nanoparticle ]

M. T. Sharbati et al., Adv. Mater. 30, 1802353 (2018)

[ Graphene ]P. Stoliar et al., Adv. Funct. Mater. 27, 11 (2017)

Pickett et al., Nat. Mater. 12, 114 (2013)

H. Lim et al. Sci. Rep. 5, 9776 (2015)

[ Mott insulators ]

S. Dutta et al., Sci. Rep. 7, 8257 (2017)

Z. Wang et al., Proc. Int. Electron Devices Meeting

(IEDM) p. 13.3.1 (2018)

[ MOSFET / CMOS ]

New material approaches

Memory

controller

The problem of uniformity

CPUMemory

SNN

A bus Buses (2.000.000.000.000.000)

Transistors Memory cells “Buses” Synapses

Equal to 285.000 Intel i9 CPUs assuming

single-transistor converters

States and Rules

Leaky integrate-and-fire Spike-timing-dependent plasticity

MemristiveBinary

States

Rules

[Co/Ni]

PtMn

In-plane magnetic field

annealing

Material system

Current, Ich

Hext = 0

S. Fukami, A. K. et al. Nat. Mater. 15, 535 (2016)

Field-free SOT switching

Current

Hall

resis

tance

Size-dependent SOT switching

5000 nm

Hall

resis

tance

Current

650 nm

Hall

resis

tance

Current

225 nm

Hall

resis

tance

Current

50 nm

In collaboration with M.

Baumgartner, G. Sala, G.

Krishnaswamy, P. Gambardella

[ETH Zurich, Switzerland]

and F. Maccherozzi, S. S. Dhesi

[Diamond Light Source, UK]

Measured at Diamond Light

Sourcce (UK) end-station I06

A. Kurenkov et al., Appl. Phys. Lett. 110, 092410 (2017)

Poster by

Gunasheel

Krishnaswamy

Size-dependent SOT switching

Variation of exchange bias

A. Kurenkov et al., Appl. Phys. Lett. 110, 092410 (2017)

-4 0 4

Co

un

ts o

f d

om

ain

s

Effective field (mT)

Out-of-plane component

-20 0 20 40

Sh

are

of

do

ma

ins

Effective field (mT)

In-plane component

MemristiveBinary

Rules: local time

Leaky integrate-and-fire Spike-timing-dependent plasticity

Pulse 1

excites

a timer Pulse 2

“reads” the timer and

changes system

accordingly

Timer

evolves in time

Local time

Switching by short pulses

-4x1011

-2x1011

2x1011

4x1011

1 s

10 s

100 ns

10 ns

1 ns

RH

all (

)

jch

(A/m2)

0.3

10-7

10-5

10-3

10-1

101

-0.3

-0.2

-0.1

0.0

0.1

0.2

0.3

P (s)

1e-2

1e-3

1e-4

RH

all (

)

Integrated time (s)

V = 6.3 Volts

Effect of temperature dynamics

10-8

10-7

10-6

10-5

10-4

5

6

7

8

9

10

0

0.1

0.2

0.3

0.4

0.5

RHall

()

Puls

e a

mp

litud

e (

V)

Pulse width (s)

10-7

10-5

10-3

10-1

101

-0.3

-0.2

-0.1

0.0

0.1

0.2

0.3

P (s)

1e-2

1e-3

1e-4

1e-5

1e-6

5e-7

3e-7

1e-7R

Hall (

)

Integrated time (s)

V = 6.3 Volts

Effect of temperature dynamics

10-8

10-7

10-6

10-5

10-4

5

6

7

8

9

10

0

0.1

0.2

0.3

0.4

0.5

RHall

()

Puls

e a

mp

litud

e (

V)

Pulse width (s)

0 10 20 300

50

100

150

200

250

300

calculated from

resistance change

fit with 4.6 s by

Troom

T (1 et /

)Te

mp

era

ture

(C

)

Time (s)

-11 0 11

-0.1

0.0

0.1

Pulse

= 10 ns

Stage temp

150°C

130°C

100°C

80°C

50°C

21°C

RH

all (

)

VPulse

(V)

Temperature-based clock

0 1 2 3

-5

0

5

0 1 2 3

25

30

35

Tem

pera

ture

(C

)V

oltage (

V)

Pulse generator output

Calculated temperature dynamics

Time (μs)

3

2

Pulse amplitude

Hall

resis

tance

1

10-8

10-7

10-6

10-5

10-4

6

7

8

9

10

"cold pair"A

mp

litude

of th

e p

uls

es |V

P| (V

)

Width of the pulses P (s)

0.00

0.04

0.08

0.12

R50ns

HallR

1ms

Hall

(Ohm)

P,-V

P

P,V

P

P,-V

P

P,V

P

1 ms

21

21

50 ns

"hot pair"

Too small

Too large

Temperature-based clock

Just right

10-8

10-7

10-6

10-5

10-4

5

6

7

8

9

10

0

0.1

0.2

0.3

0.4

0.5

RHall

()

Puls

e a

mp

litud

e (

V)

Pulse width (s)

Synaptic functionality

-5 -1 0 1 5-0.2

-0.1

0.0

0.1

0.2

post

Delay (s)

R

Ha

ll (

)

175 ns

9.2V

175 ns

9.2V

pre

A. Kurenkov et al. Adv.Mater. 31, 1900636 (2019)

Spike-timing-dependent

plasticity in rat hippocampal

synapses

G. Bi, M. Poo, J. Neurosci. 18, 10464 (1998)

Synaptic learning

E. Izhikevich, Cerebral Cortex 17, 10 (2007)

«Increase of synaptic

strength by consistent

coincidental firing of the

pre- and post-neurons»

0.01 0.1 1 10

0

1

0 ns 800 ns

0 ns 800 ns

0 ns 800 ns

100

101

102

103

104

0

1

0.0 ms 1.5 ms

0.0 ms 1.5 ms

0.0 ms 1.5 ms

npulses

= 104

fin (MHz)

Sw

itch

ing

pro

ba

bili

ty

Vp 777 mV,

p 100 ns V

p 777 mV,

p 100 ns

fin = 8 MHz

Sw

itch

ing

pro

ba

bili

ty

npulses

L. Li et al. eLife 6, 23612 (2017)

Simulation-intensity-dependent firing probability in

the artificial neurons

Neuronal functionality

Stimulation-intensity-

dependent firing probability in

a mice neuron

A. Kurenkov et al. Adv.Mater. 31, 1900636 (2019)

Coincidence detection

AB

Outcome 1 if A and B happen within a time

window

Outcome 2 if A and B are more separated

-200 -100 0 100 200

0

1

(ns)

Psw

25 ns, 999 mV

100 ns, 999 mV

100 ns, 777 mV

175 ns, 777 mV

Sound localization

T. Franken et al., Nat. Neurosci. 18,

444 (2015)

Δτ

-Δτ

CD1

CD2

CD3

Conclusions

• Large-scale and efficient spiking neural networks can’t rely on CMOS

• Uniformity of new-materials synapses and neurons is crucial (because of the

dense and highly interconnected nature of spiking networks)

• AFM/FM devices provide a uniform and

scalable solution with all the benefits of

spintronics (speed, endurance, scalability,

3D stacking)

A portion of this work was supported by JSPS

KAKENHI 17H06093, 18KK0143 and 19K15428,

JST-OPERA, JSPS Core-to-Core Program and

Cooperative Research Projects of RIEC.

More details in

A. Kurenkov et al. Advanced Materials 31, 1900636 (2019)

Material requirements for a hardware SNN basis:

• Tunable switching behavior: binary and memristive

• Presence of dynamics that can be excited and read out