ALTERNATIVES TO THE VON NEUMANN MODEL, VOL. 1, 15, MAY 2014 1

Alternatives to the Von Neumann ModelDylan Ayrey, Joshua Pueschel

Abstract—This document gives an overview of commonly used computer architectures that dont strictly follow the VonNeumann model. Among the Von Neumann alternatives out there, this document will mostly stay focused on HarvardArchitectures, Modified Harvard Architectures, and MIMD (multiple instruction multiple data) architectures.

Index Terms—Computer Architecture, Von Neumann, Harvard Architecture

F

1 INTRODUCTION

IN the late 1940s the United States Govern-ment asked Harvard and Princeton univer-

sities to develop a computer architecture thatwould be used to compute distances of Navalartillery shells for the purpose of a self de-fense application.[1] Two separate architecturescame out of this endeavor. From Harvard camethe Harvard architecture which would laterbecome a base for many Modified Harvardarchitectures. A few years later from Princetona man named John Von Neumann took thingshe learned from the Harvard model and cameup with the Princeton architecture which isotherwise known today as the Von Neumannarchitecture or the Von Neumann model. Thesetwo architectures became our current indus-try standards. They are the basis for all oftodays microprocessors and microcontrollers.Other architectures would come along as wellsuch as neural network architectures.[2] Theseother architectures have currently been gearedmore towards academic research than for com-mercial production. Due to that, this articlewill specifically target the Princeton, Harvard,and Modified Harvard architectures. Throughexamining them a list of advantages and dis-advantages for each of them can be compiled.

2 JOHN VON NEUMANN

John Von Neumann (1903 - 1957) was a math-ematician and computer scientist who is notedfor developing the Princeton architecture. In1944 Neumann was deeply involved with theManhattan Project. He spent a great amount of

time working with the Harvard Mark I usingit to make calculations for nuclear bombs[15].

Fig. 1. John Von Neumann

2.1 The Von Neumann model

After the war Neumann published a paper in1946 on his requirements for a general purposecomputer. From these requirements a computerwas eventually constructed in 1948. It wascalled the Manchester Mark I, designed andconstructed at Manchester University in Eng-land. It ran at .00083 MIPS (Million Instructionsper Second) which today would be consideredslow[5]. Neumann divided the computer intothree main blocks. These were the central pro-cessing unit (CPU), memory, and input/outputdevices (I/O). Both Memory and I/O have asingle data bus connecting them to the CPU.This will become an important distinction to

ALTERNATIVES TO THE VON NEUMANN MODEL, VOL. 1, 15, MAY 2014 2

make when later it will be compared to theHarvard Architecture.

Fig. 2. Von Neumann Model

One of the more important aspects of thismodel that is being focused on in this articleis the link between the processor and memory.Looking at a finer diagram of Fig 2.

Fig. 3. Communication between the processingunit and memory

From Fig 2, it can be seen that the processoris directly linked to the memory block througha single data bus [6]. This provides a restrictionthat is known as the Von Neumann Bottleneck.Instructions and data moving through a singlebus slows down the computer. There are someways to modify the architecture so that thisproblem is somewhat avoided. Creating the

computer so that the operations are performedin parallel is one solution. Another solutionis to separate the memory bus into two ormore busses. One bus would be for instructionsand the others would be for data [7]. Overthe years, a number of computers have beenclaimed to be ”non-von Neumann,” and manyhave been at least partially so. More and moreemphasis is being put on breaking away fromthis traditional architecture in order to achievemore usable and more productive systems. Theexpectations for the fifth generation systemsseem to require that substantially new archi-tectures be evolved, and that both hardwareand software be freed from the limitations ofthe von Neumann architecture [4]. A computerrunning on the Von Neumann model must alsoadhere to these following steps.

Fig. 4. Lifecycle of an Instruction

Fig 3 shows the lifecycle of an instruction.Adhering to the Von Neumann model, a in-struction must complete before the next in-struction begins. In processors today, pipelin-ing is a method used to enable more instruc-tions to be fired off while still adhering to

ALTERNATIVES TO THE VON NEUMANN MODEL, VOL. 1, 15, MAY 2014 3

this principle. Most modern processors due notadhere to a strictly Von Neumann model. Mostmodern processors follow it in a general sense,using other optimizations to gain speed-ups.

3 WHAT IS NON VON NEUMANN?

Non Von Neumann Architectures typically arearchitectures that are mostly suited for parallelcomputation [11]. The advantages of using anarchitecture that is highly parallelised is thatthere is a massive speed increase. The problemis that programmability on these types of ma-chines is very low. The following are examplesof Non Von Neumann architectures:

1) Analog Computers2) Optical Computers3) Quantum Computers4) Cell Processors5) Neural Networks6) MIMD architecture

One non-Von Neumann computer was de-veloped by NEC in 1984 [11]. It was an im-age pipelined processor called uPD7281D. Itspurpose was for image processing. The reasonit was considered non-Von Neumann was be-cause the pipeline could simultaneously pro-cess many different commands, breaking theVon-Neumann model.

4 PIPELINING VS PARALLELISM

The Von Neumann model requires that an in-struction completely go through its entire life-cycle before another function is fetched. Thisis quite slow and so today’s processors do notimplement this. Most of them do implementpipelining. This in and of itself breaks theVon Neumann model but it is not as large amodification as true parallelism is. There aretwo figures that can be used to show the maindifference between pipelining and parallelism:

Fig. 5. Parallelism example

Fig. 6. Pipeline example

Pipelining is an optimization that does notbreak the Von Neumann model at a higherlevel of abstraction and it provides a speedincrease. It is much harder to do this in parallel.Loops are a unique case that can currently

5 HARVARD ARCHITECTURE

The Harvard architecture came around inthe 1940s from IBM under leadership fromHoward Aiken. The term Harvard architec-ture came from the Harvard Mark I relaycomputer. This computer stored instructions

ALTERNATIVES TO THE VON NEUMANN MODEL, VOL. 1, 15, MAY 2014 4

onto punched tape with each instruction be-ing twenty-four bits wide. Data was stored inelectro-mechanical counters [8]. This was thebasis for the Harvard architecture. The dataand instructions were separated in memory.

Fig. 7. The Harvard Architecture

Figure four illustrates the separation of thedata memory and the program memory. Look-ing closer, in a strictly Harvard Architecture,the program memory is Read Only Memory(RAM). The data memory however is mutable.This method of splitting up the data and theinstructions is quite useful due to the paral-lelism that is created. More instructions can beexecuted in a given time compare to the Prince-ton architecture. An example of the differencebetween these two architectures can be broughtup through the example of a read instruction[1]. The Princeton architecture would fire it offas follows:

1) Cycle 1a) Read instruction

2) Cycle 2a) Read the data from memory

The Harvard architecture would do the fol-lowing for the same instruction

1) Cycle 1a) Finish previous instructionb) Read current instruction

2) Cycle 2a) Read the data from memoryb) Read next instruction

The advantages of the Harvard architectureare fairly straightforward. The parallelism al-lows for instruction fetches and data accessesto use separate data buses thus allowing the

Harvard architecture to be faster than a strictlyVon Neumann machine [9]. There are also otherarchitectures that exist such as the Super Har-vard architecture which is known for its in-creased throughput over the standard Harvardarchitecture. SHARC (Super Harvard Architec-ture Single-Chip Computer) DSPs make useof this architecture.[3] The SHARC architecturemakes use of having an extended cache mem-ory through which instructions can be storedthat will be reused. It also allows the programbus to be used for the transfer of data mem-ory. Through the use of the extended cache,both buses can then become free for fetchingoperands. The reason that this extended cacheworks for DSPs is because the DSPs spendmuch of their execution time in loops, passingthe same instructions from memory to the CPUover and over again. Currently today, mostmodern uses of a Harvard architecture comefrom Digital Signal Processors (DSPs). This isbecause one of the largest bottlenecks for DSPsis transferring data to and from memory [10].Many DSPs contain a need to have data, fil-ter coefficients, and program instructions beingaccessed in parallel to optimize the speed ofthe processor. The SHARC architecture is onecommon solution to the DSP problem:

Fig. 8. SHARC Architecture

Fig 5 shows an illustration of the SHARCarchitecture. The modifications from the stan-dard Harvard Architecture can be seen fromthis picture. One of the most notable differencesis the instruction cache that is contained inthe CPU [3]. The reasons for this being an

ALTERNATIVES TO THE VON NEUMANN MODEL, VOL. 1, 15, MAY 2014 5

advantage for DSPs is stated above . Apartfrom the SHARC architecture and the Harvardarchitecture, there is another way that is com-monly used to implement DSPs. The goal isto allow multiple memory accesses in a singleinstruction cycle. This can be done using amodified Von Neumann architecture. All thatmust be done is to speed up the memory clockfaster than a single instruction cycle [10]. By do-ing this you achieve the goal: Multiple memoryaccesses can occur in a single instruction cycle.

5.1 Modified Harvard Architecture

Modified Harvard Architectures are ones thatare widely used today. The architectures gener-ally relax the strict separation that the Harvardarchitecture has between instructions and data.The typically modified architectures still main-tain concurrent access to two memory buses.

Fig. 9. Cache hierarch of the AMD Athlon 64CPU

The above figure six shows how the AMDAthlon 64 CPU makes use of a Modified Har-vard Architecture. There is a split L1 cache be-tween data and instructions. This is a commonuse in most of todays Architectures. A CPUrunning from the split cache acts on a purelyHarvard Architecture.

Another architecture that falls under theModified Harvard Architecture is the ARMprocessor. This can be seen as follows:

Fig. 10. Arm architecture cache diagram

This architecture makes use of the same ad-vantages that the AMD Athlon 64 CPU makesuse of. There is a split cache for instructionsand data.

6 MIMD ARCHITECTUREThe Multiple Instruction stream, Multiple Datastream architecture has independent proces-sors that operate on separate data concurrently.Each of the processors has access to its ownprogram memory or to program memory. Like-wise, each of the processors also has access toits own data memory or to data memory [12].One of the easiest problems to see that arisesfrom this is that there needs to be a way for theprocessors to pass information to each other.There also needs to be a way for the processorsto load their program and data memories.

Fig. 11. The MIMD Architecture

There are two types of MIMD processors thatare classified. They are known as Centrelized

ALTERNATIVES TO THE VON NEUMANN MODEL, VOL. 1, 15, MAY 2014 6

Memory and Distributed Memory. Centralizedmemory has shared memory that is located ata centralized location. Distributed memory iswhen memory is distributed to each processor[14]. There are many design problems that arisewhen creating an architecture such as this one.These issues include:

1) Processor design2) Physical organization3) Interconnection structure4) Inter-processor communication protocols5) Memory hierarchy6) Cache organization and coherency7) Operating system design8) Parallel programming languages9) Application software techniques

7 ARTIFICIAL NEURAL NETWORK AR-CHITECTURE

Artificial neural networks have been the topicof focus for computer scientists and neuro-scientists for many years, going all the wayback to the early 1960s[16]. Artificial NeuralNetworks are a technology which consist ofinput nodes, center processing nodes which aretrained and output nodes. This is illustrated inFig 9.

Fig. 12. Artificial Neural Network

One of the earlier hardware implementationsof an an Artificial Neural Network was the

pRAM-256 which was finished in 1998[17]. Thischip was designed to take basic inputs andbe trained with rewards and punishment. Adiagram of this process can be seen in Fig 10.

Fig. 13. pRAM-256 Reward/Penalty diagram

It had an internal pseudo random numbergenerator which it used to help develop itselfand produce outputs. These outputs would bemodified and adjusted based on the reward orpunishment values. The full architecture dia-gram for the pRAM-256 can be seen in Figure11.

Fig. 14. pRAM-256 Architecture diagram.

Artificial Neural Networks remain the fo-cus of a lot of modern day research and de-velopment. IBM is currently working with abranch of the US military and a number ofacademic partners to develop an Artifical Neu-ral Network chip. The name of this project isSyNAPSE (Systems of Neuromorphic AdaptivePlastic Scalable Electronics)[19]

ALTERNATIVES TO THE VON NEUMANN MODEL, VOL. 1, 15, MAY 2014 7

8 CONCLUSION

Though the Von Neumann model is consideredthe basis of all modern processors, there aremany examples of violations and adaptationsof the Von Neumann model. As the field growsthe gap from pure Von Neumann widens.Lessons from the pure Von Neumann modelwill continue to be applied to modern Arche-tectures but it is unlikely to see a reappearanceof a pure Von Neumann archetecture.

REFERENCES[1] ”Basic Architectures of Microcontrollers” Internet:

http://nptel.iitk.ac.in/courses/Webcourse-contents/IIT-KANPUR/microcontrollers/micro/ui/Coursehome1 2.htm [May, 2014]

[2] Bogdan M. Wilamowski, ”Neural Network Architec-tures” Internet: http://www.eng.auburn.edu/∼wilambm/pap/2011/K10149 C006.pdf Sept. 9, 2012 [May, 2014]

[3] Steven W. Smith, Ph.D, ”The Scientist and Engineer’sGuide to Digital Signal Processing” Internet: http://www.dspguide.com/ch28/3.htm [May, 2014]

[4] Sharp, J. A. Data Flow Computing. Ellis Horwood Limited,Chichester, England. 1985

[5] H. Norton Riley, ”The von Neumann Architecture of Com-puter Systems” Internet: http://www.csupomona.edu/∼hnriley/www/VonN.html Sept, 1987 [May, 2014]

[6] ”The von Neumann Computer Model” Internet:http://www.c-jump.com/CIS77/CPU/VonNeumann/lecture.html [May, 2014]

[7] Devlin, ”John von Neumann: The Father of the Mod-ern Computer” Internet: http://www.maa.org/externalarchive/devlin/devlin 12 03.html Dec, 2003 [May, 2014]

[8] CC-BY-SA, ”Harvard architecture” Internet:http://www.princeton.edu/∼achaney/tmve/wiki100k/docs/Harvard architecture.html [May, 2014]

[9] ”The Harvard Architecture” Internet: http://cmsc411.com/computer-design-fundamentals/harvard-architecture [May, 2014]

[10] Bores, ”Introduction to DSP - DSP processors: memoryarchitectures” Internet: http://www.bores.com/courses/intro/chips/6 mem.htm [May, 2014]

[11] ”Non von Neumann Architectures” Internet: http://www.dnull.com/cpu/ [May, 2014]

[12] ”MIMD Architecture” Internet: http://siber.cankaya.edu.tr/ozdogan/GraduateParallelComputing.old/ceng505/node8.html [May, 2014]

[13] Shikha Gambhir, Hong Le, and Brenda Leung,”Introduction to MIMD Architectures” Internet:http://www.cs.umd.edu/class/fall2001/cmsc411/projects/MIMD/mimd.html [May, 2014]

[14] Zebo Peng, ”MIMD Architectures” Internet: https://www.ida.liu.se/∼TDTS08/lectures/12/lec9.pdf Nov. 23rd,2012 [May, 2014]

[15] ”John von Neumann” Internet: http://www.atomicarchive.com/Bios/vonNeumann.shtml [May,2014]

[16] B. Widrow, W. H. Pierce, J. B. Angell, ”BIRTH,LIFE, AND DEATH IN MICROELECTRONIC SYSTEMS”Internet: http://www-isl.stanford.edu/∼widrow/papers/j1961birthlife.pdf May 30, 1961 [May, 2014]

[17] ”John von Neumann: The Father of the Modern ComputerpRAM-256 VLSI Neural Network Processor pRAM artifi-cial neuron with learning in hardware” Internet: http://clarkson.me.uk/academic/pram-data.pdf Mar, 1998 [May,2014]

[18] ”Synaptic Connections” Internet: http://clarkson.me.uk/academic/pram a21.htm [May, 2014]

[19] Jason Mick, ”IBM Unveils Latest Version ofSpiking Neuron-Based Brain Chips” Internet: http://www.dailytech.com/IBM+Unveils+Latest+Version+of+Spiking+NeuronBased+Brain+Chips/article33135.htmAug. 9, 2009 [May, 2014]