The Freeman Model as an Associative Memory:

Application to Static Pattern Recognition

Mark D. Skowronski, John G. Harris, and Jose C. PrincipeComputational NeuroEngineering LabElectrical and Computer EngineeringUniversity of Florida, Gainesville, FL

April 25, 2004

2

Introduction

This work funded by the Office of Naval Research grant N00014-1-1-0405

Freeman’s Reduced KII Network

• Freeman model fundamentals• Model hierarchy• Associative memory• Experiments• Conclusions

3

Freeman Model

Hierarchical nonlinear dynamic model of cortical signal processing from rabbit olfactory neo-cortex.

Reduced KII (RKII) cell (stable oscillator)

Q(m)Kabggb)(agab

1

IQ(g)Kabmmb)(amab

1

mg

gm

K0 cell, H(s) 2nd order low pass filter

Q(x)

4

RKII Network

High-dimensional, scalable network of stable oscillators.Fully connected M-cell and G-cell weight matrices (zero diagonal).

Capable of several dynamic behaviors:• Stable attractors (limit cycle, fixed point)• Chaos• Spatio-temporal patterns• Synchronization

Generalization Associative Memory

5

Oscillator Network

Synchronization Through Stimulation (STS)

Two regimes of operation as an associative memory of binary patterns:

Energy Readout

Network weights for each regime set by outer product rule variation and by hand.

M. D. Skowronski and J. G. Harris, Phys. Rev. E, 2004 (in preparation)

6

Associative MemoryInput Output Input Output

Partial:14/22 34/12

Noisy:13/25 31/21

Spurious:22/26 24/22

Full:0/30 30/0

Hamming:“zero”/“one”

7

Two-Class Case

ASR with RKII Network

• \IY\ from “she”• \AA\ from “dark”• 10 HFCC-E coeffs.

converted to binary• Energy readout RKII

associative memory• No overlap between

learned centroids

Classifier \IY\ \AA\ % Correct

2nd order, \IY\ 2705 0 99.9

continuous \AA\ 8 4340

2nd order, \IY\ 2701 4 98.4

binary \AA\ 110 4238

1st order, \IY\ 2658 47 93.7

binary \AA\ 394 3954

RKII, \IY\ 2593 6 87.3

exact \AA\ 202 3564

RKII, \IY\ 2666 39 92.7

Hamming \AA\ 479 3869

RKII associative memory limited to 1st order, binary performance due to preprocessing restrictions.

8

ASR with RKII Network

Three-Class Case• \IY\ from “she”• \AA\ from “dark”• \AE\ from “ask”• 18 HFCC-E coeffs.

converted to binary• Energy-based RKII

associative memory• Variable overlap between

learned centroids

Overlap controlled by binary feature conversionMore overlap more spurious outputs

9

• Demonstrated static pattern classification using RKII associative memory,

• Oscillator network allows for synchronization,• Associative memory limited by binary feature

conversion and 1st order statistics,• Same issues as Hopfield associative memory:

spurious outputs, capacity, overlap,• Training by variation of outer product rule and hand

tuning.

Conclusions